You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-14 - 01:11

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Sun Sep 13, 2026 12:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
611921660,4sleep32123799cyclictest09:16:553
96612940,2sleep1221ktimersoftd/111:20:041
21234998681,3cyclictest0-21swapper/011:32:350
21234998681,3cyclictest0-21swapper/011:32:350
21236997718,23cyclictest18488-21sh11:30:332
21236997718,23cyclictest18488-21sh11:30:332
4562760,2sleep10-21swapper/110:29:111
97092750,2sleep20-21swapper/210:40:362
2123499731,57cyclictest1777-21fschecks_time10:30:380
207522673,14sleep20-21swapper/207:07:372
55972660,1sleep00-21swapper/009:15:590
265842660,2sleep20-21swapper/207:25:212
2123499661,49cyclictest0-21swapper/011:40:460
2123499656,44cyclictest32035-21diskmemload09:25:270
21234996558,5cyclictest0-21swapper/009:12:420
21234996548,3cyclictest0-21swapper/010:00:310
21234996544,9cyclictest26774-21ssh11:00:410
2123499642,35cyclictest25956-21ssh10:20:380
2123499641,54cyclictest0-21swapper/008:10:050
207462642,14sleep10-21swapper/107:07:321
165532640,2sleep30-21swapper/310:09:253
2123499635,45cyclictest16203-21systemd-journal10:40:380
2123499631,45cyclictest24819-21cron09:40:200
21234996311,50cyclictest0-21swapper/009:45:210
2123499626,38cyclictest26405-21latency_hist11:40:200
21234996233,15cyclictest19476-21diskmemload12:10:400
21234996233,12cyclictest28407-21ls12:20:470
2123499622,49cyclictest0-21swapper/010:28:060
2123499615,46cyclictest6603-21ssh12:34:580
21234996131,7cyclictest7850irq/56-dwc_otg_10:50:200
2123499610,43cyclictest0-21swapper/011:20:310
21235996040,11cyclictest31389-21sh12:25:341
21234996046,4cyclictest19599-21sh10:12:250
2123499604,52cyclictest101ktimersoftd/012:20:210
21234996036,14cyclictest15808-21latency_hist08:25:210
21234996030,15cyclictest0-21swapper/010:05:310
2123499591,33cyclictest0-21swapper/011:25:310
21234995912,43cyclictest101ktimersoftd/008:30:420
2123499590,42cyclictest11140-21cat12:40:210
2123499588,6cyclictest0-21swapper/007:25:330
2123499584,51cyclictest101ktimersoftd/009:25:200
21234995836,8cyclictest0-21swapper/007:50:380
21234995836,8cyclictest0-21swapper/007:50:380
21234995832,9cyclictest18057-21timerwakeupswit09:30:510
21234995825,16cyclictest0-21swapper/012:00:200
2123499581,37cyclictest30516-21ssh09:45:520
21512570,4sleep2301ktimersoftd/209:12:232
2123499577,40cyclictest0-21swapper/011:10:370
21234995719,12cyclictest7850irq/56-dwc_otg_10:55:320
2123499571,46cyclictest0-21swapper/007:45:430
66332560,2sleep00-21swapper/009:56:170
242602560,3sleep30-21swapper/310:18:223
2123599565,20cyclictest0-21swapper/111:25:311
2123499565,37cyclictest22095-21swap07:10:460
21234995638,9cyclictest101ktimersoftd/011:54:570
21234995635,10cyclictest21830-21latency_hist10:55:210
21234995635,10cyclictest21830-21latency_hist10:55:210
21234995633,11cyclictest11788-21sshd12:00:470
21234995633,11cyclictest11788-21sshd12:00:470
21234995629,3cyclictest7950irq/56-dwc_otg_12:25:310
2123499562,43cyclictest0-21swapper/007:30:350
2123499556,29cyclictest21620-21aten_rbpower_po10:15:330
21234995529,5cyclictest22432-21unixbench_multi08:40:490
2123499552,40cyclictest11638-21head08:10:500
256062540,3sleep10-21swapper/110:59:591
2123699548,7cyclictest0-21swapper/210:55:322
2123499547,33cyclictest24813-21ls08:50:200
2123499541,37cyclictest15728-21ps12:05:450
2123499537,40cyclictest0-21swapper/008:50:330
21234995328,9cyclictest31509-21unixbench_singl11:05:480
2123499532,41cyclictest0-21swapper/009:00:430
2123499532,37cyclictest0-21swapper/011:48:090
2123499531,43cyclictest0-21swapper/009:50:440
2123499531,38cyclictest0-21swapper/008:25:420
21234995234,5cyclictest8698-21rm11:18:170
21234995232,8cyclictest0-21swapper/008:20:200
21234995232,11cyclictest0-21swapper/010:37:490
21234995229,5cyclictest30384-21wget07:35:350
207332523,42sleep30-21swapper/307:07:223
2123499512,39cyclictest0-21swapper/009:05:390
2123499512,39cyclictest0-21swapper/009:05:390
2123499511,39cyclictest0-21swapper/008:39:100
2123499511,39cyclictest0-21swapper/008:39:100
2123499502,35cyclictest0-21swapper/008:55:320
21236994913,19cyclictest301ktimersoftd/211:20:342
2123499491,37cyclictest6869-21/usr/sbin/munin08:00:400
2123699487,25cyclictest0-21swapper/208:00:372
2123699487,24cyclictest0-21swapper/207:55:312
2123499481,37cyclictest0-21swapper/007:20:420
2123499472,35cyclictest0-21swapper/007:56:450
2123499471,39cyclictest0-21swapper/007:15:380
2123799466,5cyclictest0-21swapper/310:40:363
2123699468,26cyclictest0-21swapper/210:45:342
2123699467,25cyclictest0-21swapper/207:45:342
201182462,7sleep00-21swapper/007:05:450
2123699457,27cyclictest0-21swapper/211:15:342
2123599456,8cyclictest4173-21ssh09:15:311
2123599451,9cyclictest13870-21aten_rbpower_en10:45:351
2123499451,30cyclictest0-21swapper/007:40:400
2123699447,24cyclictest0-21swapper/208:30:342
2123699438,11cyclictest7850irq/56-dwc_otg_07:25:322
2123699435,26cyclictest17465-21ssh10:10:382
2123599437,23cyclictest6637-21ssh11:55:341
2123699427,5cyclictest7850irq/56-dwc_otg_07:10:492
2123699427,25cyclictest0-21swapper/209:15:332
2123699427,22cyclictest0-21swapper/208:20:332
2123699427,22cyclictest0-21swapper/207:30:322
2123699426,26cyclictest32035-21diskmemload10:50:372
2123699426,26cyclictest32035-21diskmemload10:50:372
21235994223,8cyclictest1210-21latency_hist07:45:201
2123599421,27cyclictest5238-21ls07:55:331
309682410,2sleep10-21swapper/109:46:321
278262410,4sleep10-21swapper/107:25:481
21237994120,4cyclictest26514-21sh08:55:203
21236994118,18cyclictest8420-21ssh09:20:312
21236994114,7cyclictest7850irq/56-dwc_otg_11:46:272
2123599415,19cyclictest7850irq/56-dwc_otg_09:30:201
2123599412,11cyclictest20634-21unixbench_multi08:35:491
2123599412,11cyclictest20634-21unixbench_multi08:35:491
34572400,2sleep30-21swapper/309:14:423
21237994019,12cyclictest37-21rcuc/308:30:483
2123699409,12cyclictest11938-21ls12:00:492
2123699409,12cyclictest11938-21ls12:00:492
2123699406,25cyclictest0-21swapper/210:30:342
21236994013,13cyclictest7950irq/56-dwc_otg_08:25:452
21236994013,13cyclictest7750irq/56-dwc_otg11:37:322
21236994010,16cyclictest7750irq/56-dwc_otg12:09:222
2123599402,8cyclictest23180-21wget12:15:361
21235994017,6cyclictest16445-1kworker/u9:111:00:311
2123799399,11cyclictest7950irq/56-dwc_otg_11:20:213
2123799392,6cyclictest18462-21irqstats10:50:443
2123799392,6cyclictest18462-21irqstats10:50:443
2123699397,22cyclictest0-21swapper/209:25:272
2123699396,24cyclictest0-21swapper/211:05:332
2123699396,21cyclictest0-21swapper/210:15:322
2123599391,9cyclictest9819-21unixbench_singl08:05:501
2123599391,6cyclictest8427-21apt08:05:201
88072381,33sleep20-21swapper/210:39:242
29042381,34sleep12912-21ssh09:13:441
21237993823,11cyclictest0-21swapper/307:20:503
2123699388,7cyclictest7950irq/56-dwc_otg_11:40:462
2123699386,17cyclictest7950irq/56-dwc_otg_11:29:262
2123599385,4cyclictest0-21swapper/107:20:331
2123599382,24cyclictest3589-21cut07:50:381
2123599382,24cyclictest3589-21cut07:50:381
2123599381,20cyclictest7950irq/56-dwc_otg_12:00:321
2123599381,20cyclictest7950irq/56-dwc_otg_12:00:321
2123599381,10cyclictest12549-21latency_hist09:25:211
136342380,1sleep30-21swapper/311:24:493
2123799371,6cyclictest8432-21sh08:05:203
21237993715,6cyclictest0-21swapper/311:00:453
21236993715,4cyclictest7750irq/56-dwc_otg10:20:482
2123599375,4cyclictest7750irq/56-dwc_otg12:30:341
21235993721,4cyclictest8320-21sh10:38:291
2123599371,3cyclictest0-21swapper/108:55:411
2123599371,10cyclictest7473-21fschecks_time12:35:371
101472370,1sleep20-21swapper/211:59:502
21237993616,6cyclictest1857550wget11:30:343
21237993616,6cyclictest1857550wget11:30:343
21237993615,5cyclictest22498-21if_wlan010:55:383
21237993614,9cyclictest1180-21runrttasks08:17:373
2123799361,18cyclictest1671-21df10:30:353
2123699367,24cyclictest7950irq/56-dwc_otg_11:10:362
2123699365,5cyclictest7850irq/56-dwc_otg_09:50:362
2123699364,8cyclictest0-21swapper/212:25:312
2123699364,4cyclictest22573-1kworker/2:2H08:55:212
2123699364,16cyclictest7850irq/56-dwc_otg_07:50:422
2123699364,16cyclictest7850irq/56-dwc_otg_07:50:422
2123699363,27cyclictest7950irq/56-dwc_otg_12:15:332
2123699362,6cyclictest301ktimersoftd/209:55:502
21236993616,9cyclictest7750irq/56-dwc_otg09:49:552
21236993613,7cyclictest7750irq/56-dwc_otg10:03:562
2123699361,2cyclictest0-21swapper/208:05:332
21236993612,15cyclictest0-21swapper/212:20:422
21236993611,9cyclictest6134-21ssh11:55:052
21236993610,14cyclictest301ktimersoftd/210:06:472
2123599363,6cyclictest1488-21wget07:45:341
2123599361,7cyclictest27156-21if_err_wlan011:40:391
2123599361,6cyclictest26010-21sh09:40:441
2123599361,4cyclictest6275-21grep09:55:491
2123599361,24cyclictest31911-21timerwakeupswit11:45:471
21235993610,15cyclictest0-21swapper/110:18:021
2123799358,4cyclictest7850irq/56-dwc_otg_10:45:383
2123799352,7cyclictest30444-21/usr/sbin/munin09:05:413
2123799352,7cyclictest30444-21/usr/sbin/munin09:05:413
2123799351,6cyclictest6321-21grep09:55:503
2123799351,5cyclictest0-21swapper/312:25:213
21237993515,5cyclictest23207-21wget12:15:363
21237993513,4cyclictest1180-21runrttasks11:26:513
21237993510,4cyclictest30140-21grep10:25:383
2123699351,6cyclictest301ktimersoftd/208:20:202
2123699351,5cyclictest0-21swapper/209:00:332
21236993513,4cyclictest7750irq/56-dwc_otg12:37:382
21236993513,4cyclictest21027-21/usr/sbin/munin09:35:402
2123599359,5cyclictest0-21swapper/110:52:541
2123599359,5cyclictest0-21swapper/110:52:541
2123599355,15cyclictest7850irq/56-dwc_otg_11:10:411
21235993513,4cyclictest7850irq/56-dwc_otg_10:40:511
21235993512,11cyclictest0-21swapper/112:20:351
2123599351,13cyclictest0-21swapper/111:36:051
21235993510,4cyclictest0-21swapper/111:30:341
21235993510,4cyclictest0-21swapper/111:30:341
2123799348,6cyclictest7950irq/56-dwc_otg_10:40:203
2123799347,7cyclictest31447-21unixbench_multi11:05:483
2123799345,5cyclictest7750irq/56-dwc_otg07:40:483
2123799344,5cyclictest14702-21ls08:20:423
21237993424,6cyclictest381ktimersoftd/312:05:313
2123799341,7cyclictest25726-21basename10:20:343
21237993417,5cyclictest5596-21sh11:15:103
2123799341,5cyclictest17885-21/usr/sbin/munin09:30:503
21237993413,5cyclictest9809-21missed_timers10:00:403
21237993410,13cyclictest0-21swapper/307:50:443
21237993410,13cyclictest0-21swapper/307:50:443
2123699344,16cyclictest7950irq/56-dwc_otg_09:41:342
2123699341,20cyclictest301ktimersoftd/212:14:532
21236993411,10cyclictest7850irq/56-dwc_otg_09:33:392
2123599349,6cyclictest7850irq/56-dwc_otg_09:50:481
2123599341,7cyclictest21655-21fschecks_time08:40:381
2123599341,7cyclictest16203-21systemd-journal10:20:341
21235993412,11cyclictest0-21swapper/109:30:431
21235993411,6cyclictest15151-21/usr/sbin/munin08:20:481
21235993411,11cyclictest221ktimersoftd/111:05:531
2123799338,4cyclictest1180-21runrttasks07:14:443
2123799334,14cyclictest7850irq/56-dwc_otg_09:50:493
21237993316,11cyclictest381ktimersoftd/311:44:303
21237993314,11cyclictest1180-21runrttasks07:36:513
2123699339,13cyclictest7950irq/56-dwc_otg_11:01:012
2123699337,6cyclictest7950irq/56-dwc_otg_08:45:432
2123699337,12cyclictest7950irq/56-dwc_otg_08:44:082
2123699336,7cyclictest7850irq/56-dwc_otg_07:35:382
2123599338,5cyclictest7850irq/56-dwc_otg_08:15:321
2123599338,15cyclictest0-21swapper/112:09:071
2123599334,6cyclictest0-21swapper/109:35:351
2123599332,5cyclictest2231-21ntp_states10:30:431
2123599332,22cyclictest0-21swapper/110:00:281
2123599331,6cyclictest10791-21fschecks_time08:10:391
21235993312,7cyclictest0-21swapper/112:14:261
2123599331,22cyclictest30690-21cut09:05:391
2123599331,22cyclictest30690-21cut09:05:391
21235993312,11cyclictest0-21swapper/110:05:341
2123599331,18cyclictest7950irq/56-dwc_otg_10:15:211
276132320,6sleep32123799cyclictest09:42:563
2123799328,5cyclictest0-21swapper/309:45:453
2123799327,14cyclictest0-21swapper/312:30:373
2123799325,5cyclictest7750irq/56-dwc_otg08:25:423
2123799324,5cyclictest7850irq/56-dwc_otg_08:45:343
2123799322,7cyclictest9849-21users08:05:513
21237993218,4cyclictest29354-21ntp_states09:00:463
2123799321,6cyclictest22589-21apt-get11:35:343
21237993214,5cyclictest3303-21cat11:50:463
21237993212,5cyclictest1454-21ssh11:49:033
2123799321,22cyclictest21453-21idleruntime09:35:393
2123799321,22cyclictest13154-21uniq09:25:393
2123699326,14cyclictest0-21swapper/207:40:332
2123699322,5cyclictest31027-21grep10:25:492
2123699322,2cyclictest0-21swapper/208:58:042
21236993211,5cyclictest10596-21/usr/sbin/munin08:10:362
2123599329,12cyclictest0-21swapper/111:55:181
2123599328,11cyclictest7950irq/56-dwc_otg_08:35:201
2123599321,8cyclictest26502-21cron08:55:201
21235993213,6cyclictest0-21swapper/107:15:471
2123799311,5cyclictest28945-21/usr/sbin/munin07:30:373
21237993112,5cyclictest9563-21timerwakeupswit09:20:473
2123799311,20cyclictest17840-21ssh10:10:423
2123699318,8cyclictest29-21rcuc/212:31:522
2123699317,5cyclictest7850irq/56-dwc_otg_07:15:302
21236993110,8cyclictest7950irq/56-dwc_otg_08:36:052
21236993110,8cyclictest7950irq/56-dwc_otg_08:36:052
2123599317,10cyclictest0-21swapper/111:20:331
2123599316,12cyclictest7950irq/56-dwc_otg_08:45:341
2123599312,16cyclictest7950irq/56-dwc_otg_07:14:461
2123799306,15cyclictest0-21swapper/312:00:343
2123799306,15cyclictest0-21swapper/312:00:343
2123799306,14cyclictest0-21swapper/311:55:343
2123799303,5cyclictest0-21swapper/312:35:463
2123799303,4cyclictest7950irq/56-dwc_otg_09:00:203
2123799302,5cyclictest0-21swapper/308:40:383
21237993020,7cyclictest21115-21latency_hist08:40:213
21237993020,7cyclictest21115-21latency_hist08:40:213
2123799301,4cyclictest10893-21if_eth008:10:403
21237993014,4cyclictest26684-21/usr/sbin/munin07:25:383
21237993011,5cyclictest31927-21latency12:25:413
21236993010,4cyclictest30856-21if_eth009:05:402
21236993010,4cyclictest30856-21if_eth009:05:402
2123599302,6cyclictest7750irq/56-dwc_otg08:25:351
2123799294,5cyclictest5900-21/usr/sbin/munin07:55:433
2123799294,10cyclictest0-21swapper/312:10:443
2123799293,6cyclictest1180-21runrttasks07:45:353
2123799291,13cyclictest7950irq/56-dwc_otg_07:17:283
2123599296,4cyclictest7750irq/56-dwc_otg07:35:421
2123599296,4cyclictest7750irq/56-dwc_otg07:30:391
2123599293,14cyclictest7950irq/56-dwc_otg_09:00:351
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional