You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-09-28 - 03:31

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Thu Sep 28, 2023 00:43:38)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2193321840,5sleep0839399cyclictest21:56:120
3110521760,4sleep1839499cyclictest20:15:511
830621360,5sleep10-21swapper/119:10:261
839599865,10cyclictest8165-21ssh00:30:472
313672850,2sleep0839399cyclictest00:17:060
237302840,1sleep10-21swapper/121:16:201
303582830,3sleep121-21rcuc/123:32:221
839399815,52cyclictest26327-21apt-get22:45:440
8393998028,47cyclictest8392-21cyclictest19:10:270
58142790,2sleep00-21swapper/023:00:530
839499785,15cyclictest18424-21ssh21:10:421
8395997612,14cyclictest8050irq/56-dwc_otg_21:05:412
839399761,63cyclictest7761-21sh00:30:240
839599743,14cyclictest7950irq/56-dwc_otg20:40:452
296352730,2sleep20-21swapper/222:06:172
193922720,3sleep319371-21ssh21:10:593
839399713,54cyclictest30454-21fschecks_count22:50:490
839399703,56cyclictest23686-21wget19:55:450
839399691,55cyclictest21623-21wc19:50:250
839399691,39cyclictest0-21swapper/020:55:450
839399691,37cyclictest0-21swapper/023:10:460
839399682,49cyclictest23179-21awk00:05:570
8393996812,41cyclictest2293-21ntpq22:55:540
839599675,11cyclictest7950irq/56-dwc_otg22:25:422
839399677,48cyclictest14627-21latency_hist22:30:250
839399670,56cyclictest22614-21fschecks_time21:15:490
839399670,49cyclictest11869-21runrttasks00:35:450
8393996510,45cyclictest15319-21fschecks_count23:55:480
147872650,2sleep10-21swapper/121:47:201
129242650,2sleep30-21swapper/323:10:263
839399646,49cyclictest0-21swapper/022:12:410
8393996415,8cyclictest3934-21if_err_eth023:40:490
839699639,11cyclictest8050irq/56-dwc_otg_21:20:423
839399637,37cyclictest0-21swapper/000:00:460
8396996224,23cyclictest8150irq/56-dwc_otg_22:41:013
839699622,14cyclictest0-21swapper/300:20:423
839399629,41cyclictest25367-21/usr/sbin/munin20:01:020
839399625,44cyclictest0-21swapper/023:26:490
839399622,46cyclictest27242-21grep21:21:000
839399621,46cyclictest0-21swapper/023:47:420
839399621,29cyclictest0-21swapper/022:20:420
8393996211,42cyclictest18982-21diskstats22:35:500
8393996211,42cyclictest18982-21diskstats22:35:500
8396996122,15cyclictest18553-21ssh21:52:083
839399616,44cyclictest0-21swapper/023:24:180
839499607,35cyclictest0-21swapper/123:20:431
8393996015,41cyclictest101ktimersoftd/020:35:590
8395995910,10cyclictest31-21ksoftirqd/221:10:432
839399599,35cyclictest32305-21rm21:28:160
839399598,41cyclictest0-21swapper/023:06:130
839399591,46cyclictest11482-21grep23:50:490
80772593,14sleep20-21swapper/219:09:512
8396995824,11cyclictest22654-21perl21:15:493
8396995821,16cyclictest30468-21fschecks_count22:50:493
8396995819,17cyclictest21960-21cpuspeed19:50:453
839399587,30cyclictest9005-21aten_rbpower_en21:40:440
839399585,38cyclictest0-21swapper/022:17:240
839399585,38cyclictest0-21swapper/022:17:240
839399585,26cyclictest0-21swapper/020:05:430
839399584,37cyclictest18575-21sh21:52:110
839399582,40cyclictest18349-21taskset21:10:350
839399581,46cyclictest6803-21chrt21:36:440
839399581,40cyclictest27415-21taskset00:11:210
839399581,40cyclictest27415-21taskset00:11:210
8396995724,14cyclictest4131-21ssh21:34:103
8396995721,14cyclictest1772-21ssh23:37:333
8396995719,13cyclictest23542-21ssh00:06:033
8393995714,26cyclictest17391-21/usr/sbin/munin23:15:530
8393995713,25cyclictest16582-21cpu21:05:440
839399571,24cyclictest7670-21fschecks_time20:40:510
8393995710,31cyclictest4272-21sh21:34:260
8396995620,13cyclictest31391-21ssh00:17:093
8396995619,22cyclictest14379-21latency_hist21:00:253
839599565,13cyclictest7950irq/56-dwc_otg20:00:452
839399568,36cyclictest24832-21chrt22:43:060
839399568,32cyclictest22001-21grep19:50:470
839399567,40cyclictest16024-21chrt19:34:360
839399567,35cyclictest17161-21ssh22:32:560
839399566,42cyclictest28004-21latency_hist22:05:250
839399562,28cyclictest0-21swapper/020:10:440
8396995525,16cyclictest4026-21ssh22:58:273
839399558,34cyclictest18027-21diskmemload22:07:330
8393995525,7cyclictest12715-21tr19:25:390
8393995511,4cyclictest0-21swapper/000:30:450
8396995423,12cyclictest10788-21ssh23:06:313
8396995420,12cyclictest13584-21ssh23:53:123
839399544,34cyclictest30211-21taskset23:32:040
839399543,40cyclictest0-21swapper/023:37:130
8393995425,7cyclictest1233-21grep20:20:590
839399541,43cyclictest4876-21tail20:31:020
8396995326,13cyclictest18489-21ssh23:16:403
8396995325,16cyclictest9602-21wc21:40:523
8396995323,12cyclictest3282-21ssh22:15:273
8396995323,12cyclictest3282-21ssh22:15:273
8396995320,11cyclictest6333-21ssh23:01:013
8396995319,18cyclictest28397-21ssh00:13:373
8396995319,18cyclictest28397-21ssh00:13:373
8396995317,9cyclictest8150irq/56-dwc_otg_20:20:243
839499533,11cyclictest0-21swapper/122:35:421
839499533,11cyclictest0-21swapper/122:35:421
8394995317,23cyclictest8150irq/56-dwc_otg_22:40:421
839399534,36cyclictest0-21swapper/020:30:250
8393995328,8cyclictest13625-21ssh21:45:580
839399532,41cyclictest0-21swapper/000:24:570
8396995226,11cyclictest18924-21cpuspeed22:35:493
8396995226,11cyclictest18924-21cpuspeed22:35:493
8396995225,17cyclictest27677-21ssh23:29:433
8396995225,11cyclictest18853-21ssh00:00:433
8396995223,16cyclictest4821-21unixbench-2d20:31:023
8396995222,11cyclictest11109-21apt-get22:25:423
8396995222,10cyclictest21521-21latency_hist19:50:253
8396995219,17cyclictest8150irq/56-dwc_otg_23:55:573
8396995218,20cyclictest21293-21ssh23:20:513
8396995218,10cyclictest8050irq/56-dwc_otg_00:30:293
839499523,12cyclictest0-21swapper/122:15:441
839499523,12cyclictest0-21swapper/122:15:441
839399525,38cyclictest0-21swapper/020:52:070
839399521,25cyclictest0-21swapper/020:45:430
8396995121,17cyclictest18027-21diskmemload23:42:473
8396995121,15cyclictest12395-21cat00:35:543
8396995120,16cyclictest30264-21ssh22:07:353
8396995120,12cyclictest9181-21/usr/sbin/munin20:45:573
8396995119,11cyclictest27344-21cat22:46:013
8396995116,23cyclictest8050irq/56-dwc_otg_22:30:483
8396995116,20cyclictest8050irq/56-dwc_otg_21:26:303
8394995115,5cyclictest8150irq/56-dwc_otg_23:35:431
839399515,35cyclictest0-21swapper/020:19:540
839399511,32cyclictest0-21swapper/021:01:000
8396995027,16cyclictest12142-21sshd19:21:533
8396995015,17cyclictest8150irq/56-dwc_otg_19:10:593
839599503,25cyclictest7950irq/56-dwc_otg00:25:422
8396994922,12cyclictest7202-21ssh22:20:323
8396994922,11cyclictest31032-21rm23:33:473
8396994921,10cyclictest7736-21ssh00:30:233
8396994919,16cyclictest27098-21ssh22:03:573
8396994917,21cyclictest10802-21diskmemload23:50:233
8396994916,17cyclictest30234-21taskset20:11:513
8396994916,11cyclictest8150irq/56-dwc_otg_20:01:333
839499497,26cyclictest0-21swapper/119:10:441
839499493,7cyclictest5160-21ssh21:35:451
8394994912,23cyclictest0-21swapper/121:05:411
839399497,32cyclictest19737-21wc19:45:240
8393994913,32cyclictest101ktimersoftd/019:37:140
8396994811,19cyclictest8150irq/56-dwc_otg_21:37:093
8393994821,13cyclictest10696-21cut19:15:500
8393994821,13cyclictest10696-21cut19:15:500
839399481,36cyclictest0-21swapper/019:24:060
8396994718,10cyclictest16054-21ssh21:49:543
839599475,5cyclictest0-21swapper/223:50:442
839499477,26cyclictest0-21swapper/120:20:441
839499473,27cyclictest8050irq/56-dwc_otg_20:25:521
301482470,4sleep231-21ksoftirqd/221:25:472
8396994622,12cyclictest19462-21seq19:42:293
8396994619,12cyclictest20511-21/usr/sbin/munin21:55:573
839499467,26cyclictest5413-21ssh23:00:471
839499467,24cyclictest0-21swapper/121:40:431
81172462,36sleep00-21swapper/019:10:220
8396994522,14cyclictest17840-21runrttasks21:08:143
8396994511,21cyclictest8150irq/56-dwc_otg_21:02:433
8396994434,7cyclictest8013-21ls20:40:563
8396994423,10cyclictest31895-21ssh22:10:273
839699441,5cyclictest8150irq/56-dwc_otg_20:25:463
839599449,19cyclictest7950irq/56-dwc_otg22:35:422
839599449,19cyclictest7950irq/56-dwc_otg22:35:422
839599447,25cyclictest0-21swapper/220:15:462
839599444,3cyclictest0-21swapper/219:40:442
839499447,22cyclictest0-21swapper/121:00:301
839699435,6cyclictest8150irq/56-dwc_otg_19:35:433
8396994326,11cyclictest23561-21munin-node19:55:423
8396994325,15cyclictest774-21ls20:20:523
8395994333,7cyclictest26052-21latency_hist22:45:252
839499434,12cyclictest0-21swapper/122:25:421
8396994218,11cyclictest15846-21taskset19:32:183
8395994224,7cyclictest8050irq/56-dwc_otg_22:30:512
8395994215,23cyclictest0-21swapper/222:55:432
839499425,10cyclictest0-21swapper/100:30:471
8396994124,12cyclictest27214-21wget20:05:443
8396994116,10cyclictest0-21swapper/319:26:473
8395994119,5cyclictest7950irq/56-dwc_otg21:45:512
839499417,23cyclictest0-21swapper/119:40:451
839499416,6cyclictest8050irq/56-dwc_otg_00:36:021
839499416,5cyclictest7950irq/56-dwc_otg21:35:261
839499414,6cyclictest0-21swapper/120:30:441
8396994024,13cyclictest10640-21if_eth019:15:483
8396994024,13cyclictest10640-21if_eth019:15:483
839599403,3cyclictest0-21swapper/222:10:462
839599403,23cyclictest0-21swapper/219:10:272
839599401,4cyclictest0-21swapper/223:25:412
839499407,23cyclictest0-21swapper/100:05:461
839599398,15cyclictest8150irq/56-dwc_otg_21:30:592
839599396,22cyclictest0-21swapper/223:40:432
839599393,5cyclictest0-21swapper/223:55:432
839599392,24cyclictest10979-21ls23:50:252
8395993915,9cyclictest21189-21rm21:55:532
839499399,18cyclictest0-21swapper/122:05:591
839499395,4cyclictest8050irq/56-dwc_otg_22:00:561
839499391,24cyclictest8150irq/56-dwc_otg_00:20:431
8396993824,11cyclictest6270-21ntp_states20:35:543
8396993824,11cyclictest11589-21wc20:50:543
839499386,21cyclictest0-21swapper/100:10:441
839499386,21cyclictest0-21swapper/100:10:441
8394993816,5cyclictest3199-21run-parts22:56:411
839499380,17cyclictest13081-21apt-get23:10:461
839599371,8cyclictest0-21swapper/219:30:452
839599371,22cyclictest0-21swapper/220:05:442
8395993710,19cyclictest0-21swapper/200:00:552
8394993730,5cyclictest10049-21ls20:45:581
8394993716,6cyclictest1457950wget19:30:471
839599367,14cyclictest8050irq/56-dwc_otg_20:50:512
839599365,15cyclictest8150irq/56-dwc_otg_23:30:512
8395993628,6cyclictest8150irq/56-dwc_otg_21:35:442
839599362,7cyclictest20721-21grep19:45:592
839599362,19cyclictest8050irq/56-dwc_otg_22:04:572
8395993616,4cyclictest25466-21sh21:19:572
839599361,20cyclictest10464-21cut19:15:462
839599361,20cyclictest10464-21cut19:15:462
8395993610,6cyclictest3740-21fschecks_time22:15:492
8395993610,6cyclictest3740-21fschecks_time22:15:492
839499364,4cyclictest0-21swapper/122:51:021
839499361,5cyclictest9201-21/usr/sbin/munin23:05:541
839499361,25cyclictest15545-21latency22:30:551
839599351,23cyclictest9803-21latency23:05:522
839599351,14cyclictest0-21swapper/222:52:042
839499357,23cyclictest0-21swapper/119:25:391
839499357,15cyclictest0-21swapper/120:50:441
839499351,24cyclictest0-21swapper/123:45:441
839499351,24cyclictest0-21swapper/122:10:481
839499351,22cyclictest7513-21ls22:20:461
839599342,8cyclictest27547-21grep22:46:032
839599342,20cyclictest0-21swapper/219:35:472
8395993421,9cyclictest17888-21kworker/2:023:00:462
8395993415,6cyclictest434-21latency_hist00:20:252
839599341,21cyclictest0-21swapper/220:55:452
839599340,14cyclictest0-21swapper/200:25:252
839499349,4cyclictest8150irq/56-dwc_otg_23:41:001
839499348,5cyclictest7950irq/56-dwc_otg21:55:581
839499343,3cyclictest7950irq/56-dwc_otg00:00:491
839499342,6cyclictest7950irq/56-dwc_otg21:25:251
839499342,5cyclictest0-21swapper/120:40:251
839499341,6cyclictest8050irq/56-dwc_otg_20:55:521
8394993413,4cyclictest0-21swapper/123:30:251
8394993411,11cyclictest0-21swapper/100:27:141
839599339,3cyclictest0-21swapper/200:06:042
839599338,6cyclictest8050irq/56-dwc_otg_23:20:592
839599336,22cyclictest0-21swapper/200:35:452
839599336,15cyclictest0-21swapper/221:00:312
839599335,7cyclictest7950irq/56-dwc_otg21:40:422
839599335,13cyclictest8050irq/56-dwc_otg_22:20:362
839599334,10cyclictest0-21swapper/200:10:412
839599334,10cyclictest0-21swapper/200:10:412
839599331,5cyclictest8150irq/56-dwc_otg_21:51:002
839599331,5cyclictest22105-21fschecks_time19:50:472
839499336,4cyclictest7950irq/56-dwc_otg19:50:411
839499335,5cyclictest8050irq/56-dwc_otg_19:56:001
839499332,15cyclictest8050irq/56-dwc_otg_20:05:251
8394993314,5cyclictest21-21rcuc/120:10:561
8394993311,5cyclictest8150irq/56-dwc_otg_23:51:011
839599325,14cyclictest8150irq/56-dwc_otg_23:11:112
839599321,4cyclictest26963-21/usr/sbin/munin21:20:572
839599321,20cyclictest29267-21cut20:10:502
839499326,6cyclictest0-21swapper/121:25:421
839499326,20cyclictest7950irq/56-dwc_otg21:52:091
839499324,14cyclictest8150irq/56-dwc_otg_23:16:001
839499321,19cyclictest30675-21timerwakeupswit00:16:011
8394993210,5cyclictest0-21swapper/100:00:091
839599317,11cyclictest8050irq/56-dwc_otg_20:20:512
839599314,14cyclictest23561-21/usr/sbin/munin19:55:572
839599313,13cyclictest8150irq/56-dwc_otg_20:35:242
8395993112,8cyclictest18027-21diskmemload23:15:472
8395993110,4cyclictest0-21swapper/223:39:062
839499314,14cyclictest8050irq/56-dwc_otg_20:05:561
839499311,21cyclictest7901-21latency20:40:541
78942312,22sleep30-21swapper/319:07:253
839499303,10cyclictest20055-21aten_rbpower_po19:45:481
839499302,8cyclictest26923-21cat22:45:551
29342300,5sleep2839599cyclictest20:26:022
839599293,3cyclictest0-21swapper/220:35:442
839499293,17cyclictest0-21swapper/119:23:171
839499292,15cyclictest0-21swapper/119:20:241
839499292,15cyclictest0-21swapper/119:20:241
839599286,4cyclictest7950irq/56-dwc_otg19:30:242
839599282,11cyclictest8150irq/56-dwc_otg_20:45:522
839599261,16cyclictest0-21swapper/219:24:162
839499261,5cyclictest7950irq/56-dwc_otg19:35:511
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional