You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-16 - 06:31

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Sun Aug 16, 2026 00:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
78599270,24cyclictest27648-21diskmemload21:18:323
78599270,23cyclictest0-21swapper/323:28:453
785992623,0cyclictest0-21swapper/321:55:273
785992623,0cyclictest0-21swapper/319:40:023
78599260,22cyclictest27034-21sh22:08:243
784992625,1cyclictest21819-21/usr/sbin/munin21:34:502
78499260,2cyclictest8003-21munin-node22:49:502
784992522,2cyclictest12759-21df00:19:522
784992520,4cyclictest10788-21ssh23:49:252
78299251,2cyclictest22-21ksoftirqd/123:29:501
78299250,24cyclictest27648-21diskmemload21:24:431
78299250,24cyclictest0-21swapper/100:21:221
78199250,24cyclictest0-21swapper/022:42:180
78199250,24cyclictest0-21swapper/022:25:220
78199250,24cyclictest0-21swapper/022:06:370
78199250,1cyclictest0-21swapper/022:17:280
78599240,24cyclictest27648-21diskmemload22:24:293
78599240,24cyclictest0-21swapper/300:01:073
78599240,23cyclictest10837-21ssh21:24:463
784992422,1cyclictest0-21swapper/222:58:142
784992421,3cyclictest0-21swapper/223:26:462
784992420,3cyclictest0-21swapper/200:11:502
78499240,23cyclictest2654-21ssh23:41:022
78499240,23cyclictest15209-21apt-get19:40:012
78499240,23cyclictest0-21swapper/223:58:432
78499240,23cyclictest0-21swapper/221:44:492
782992423,0cyclictest0-21swapper/123:07:481
782992422,1cyclictest0-21swapper/121:30:001
782992422,1cyclictest0-21swapper/120:29:531
78299240,2cyclictest4861-21rm22:45:291
78299240,24cyclictest27648-21diskmemload00:35:581
78299240,24cyclictest0-21swapper/122:04:261
78299240,24cyclictest0-21swapper/121:47:011
78299240,24cyclictest0-21swapper/121:47:011
78299240,23cyclictest27648-21diskmemload22:14:571
78299240,23cyclictest15209-21apt-get19:40:021
78299240,23cyclictest10068-21perf20:35:001
78299240,23cyclictest0-21swapper/121:37:311
781992423,1cyclictest62550irq/127-eno121:30:310
781992421,2cyclictest8156-21munin-plugin-st00:15:010
781992421,2cyclictest0-21swapper/023:59:200
78199240,24cyclictest0-21swapper/023:48:470
78199240,24cyclictest0-21swapper/021:28:350
78199240,24cyclictest0-21swapper/000:38:350
78199240,23cyclictest27648-21diskmemload23:31:090
78199240,23cyclictest27648-21diskmemload00:28:100
78199240,23cyclictest26450-21sh00:30:370
78199240,1cyclictest0-21swapper/023:14:510
78199240,0cyclictest27648-21diskmemload22:33:490
78199240,0cyclictest27648-21diskmemload22:33:490
78199240,0cyclictest0-21swapper/021:57:280
78199240,0cyclictest0-21swapper/000:09:280
78199240,0cyclictest0-21swapper/000:09:280
785992323,0cyclictest4123-21ssh21:47:013
785992323,0cyclictest4123-21ssh21:47:013
785992323,0cyclictest0-21swapper/322:36:243
785992323,0cyclictest0-21swapper/322:04:263
785992323,0cyclictest0-21swapper/321:33:393
785992323,0cyclictest0-21swapper/300:35:583
785992323,0cyclictest0-21swapper/300:21:223
785992322,1cyclictest27648-21diskmemload23:49:063
78599230,2cyclictest2534-21sh23:12:463
78599230,2cyclictest11047-21rm22:51:093
78599230,23cyclictest0-21swapper/322:42:413
78599230,23cyclictest0-21swapper/322:11:163
78599230,23cyclictest0-21swapper/321:53:413
78599230,23cyclictest0-21swapper/321:37:393
78599230,22cyclictest28548-21ssh21:10:203
78599230,22cyclictest20999-21diskmemload00:25:533
78599230,22cyclictest20969-21ssh22:31:163
78599230,22cyclictest20969-21ssh22:31:163
78599230,22cyclictest19376-21sh23:55:423
78599230,1cyclictest0-21swapper/322:45:293
78599230,0cyclictest0-21swapper/323:37:543
78599230,0cyclictest0-21swapper/300:15:203
784992323,0cyclictest0-21swapper/222:40:412
784992323,0cyclictest0-21swapper/200:26:102
784992322,1cyclictest27648-21diskmemload21:17:472
78499230,2cyclictest762-21sh00:37:162
78499230,23cyclictest0-21swapper/223:14:152
78499230,23cyclictest0-21swapper/222:38:142
78499230,22cyclictest9569-21ssh21:51:502
78499230,22cyclictest7138-21sh23:16:172
78499230,22cyclictest29827-21ssh00:05:012
78499230,22cyclictest29827-21ssh00:05:012
78499230,22cyclictest26256-21sh00:02:282
78499230,22cyclictest21256-21sh22:03:012
78499230,22cyclictest0-21swapper/221:10:462
78499230,1cyclictest0-21swapper/221:40:002
78499230,0cyclictest0-21swapper/223:22:542
78499230,0cyclictest0-21swapper/222:20:372
78499230,0cyclictest0-21swapper/222:11:252
782992323,0cyclictest0-21swapper/123:21:451
782992323,0cyclictest0-21swapper/123:04:431
782992323,0cyclictest0-21swapper/122:30:021
782992323,0cyclictest0-21swapper/122:30:021
782992323,0cyclictest0-21swapper/100:33:081
782992322,0cyclictest0-21swapper/123:44:391
782992321,1cyclictest25354-21kworker/u8:222:06:231
782992321,1cyclictest0-21swapper/123:51:271
782992321,1cyclictest0-21swapper/123:46:361
782992321,1cyclictest0-21swapper/120:55:001
78299230,23cyclictest15959-21sh22:27:061
78299230,23cyclictest0-21swapper/123:11:401
78299230,23cyclictest0-21swapper/122:20:281
78299230,23cyclictest0-21swapper/121:33:391
78299230,23cyclictest0-21swapper/121:19:111
78299230,23cyclictest0-21swapper/121:11:401
78299230,23cyclictest0-21swapper/100:25:511
78299230,22cyclictest26780-21ssh00:03:081
78299230,22cyclictest19340-21ssh23:55:391
78299230,22cyclictest12203-21sh22:52:481
78299230,22cyclictest0-21swapper/122:39:511
78299230,22cyclictest0-21swapper/121:43:041
78299230,22cyclictest0-21swapper/100:15:061
78299230,0cyclictest151rcu_preempt22:15:081
78299230,0cyclictest0-21swapper/122:57:081
78299230,0cyclictest0-21swapper/121:55:271
78299230,0cyclictest0-21swapper/121:53:401
781992323,0cyclictest0-21swapper/022:35:080
781992322,1cyclictest62550irq/127-eno121:23:550
781992322,1cyclictest62550irq/127-eno100:24:300
781992322,1cyclictest27648-21diskmemload22:11:050
781992321,1cyclictest4352-21sh21:18:280
781992321,1cyclictest0-21swapper/023:41:350
781992321,1cyclictest0-21swapper/023:17:160
781992321,1cyclictest0-21swapper/021:42:430
781992321,1cyclictest0-21swapper/000:02:410
781992320,2cyclictest0-21swapper/023:23:490
78199230,2cyclictest12608-21sendmail20:40:000
78199230,22cyclictest5906-21sh22:46:540
78199230,22cyclictest28480-21ssh23:06:370
78199230,22cyclictest27648-21diskmemload22:53:510
78199230,22cyclictest27005-21munin-run23:05:000
78199230,22cyclictest0-21swapper/023:51:420
78199230,22cyclictest0-21swapper/021:36:480
78199230,21cyclictest27648-21diskmemload23:36:570
78199230,0cyclictest0-21swapper/023:26:370
78199230,0cyclictest0-21swapper/021:12:180
785992222,0cyclictest5158-21ssh22:17:393
785992221,1cyclictest5860-21apt-get20:29:533
785992221,1cyclictest27648-21diskmemload00:06:013
785992221,1cyclictest27648-21diskmemload00:06:013
785992219,2cyclictest20457-21turbostat20:55:003
78599220,2cyclictest6872-21diskmemload23:15:583
78599220,22cyclictest27648-21diskmemload23:43:293
78599220,22cyclictest27648-21diskmemload00:10:033
78599220,22cyclictest0-21swapper/323:07:183
78599220,22cyclictest0-21swapper/323:01:273
78599220,22cyclictest0-21swapper/321:26:013
78599220,21cyclictest12120-21sh23:20:173
78599220,21cyclictest0-21swapper/320:14:503
78599220,1cyclictest5231-21perf20:25:013
78599220,1cyclictest16506-21diskmemload22:56:103
78599220,0cyclictest27648-21diskmemload23:52:093
78599220,0cyclictest0-21swapper/300:30:043
784992222,0cyclictest27648-21diskmemload22:05:532
784992222,0cyclictest0-21swapper/222:29:362
784992221,1cyclictest21741-21kworker/u8:023:05:142
784992220,1cyclictest30997-21perf00:35:002
784992220,1cyclictest15274-21rm00:20:332
784992220,1cyclictest10267-21ssh22:50:102
78499220,22cyclictest27648-21diskmemload21:25:532
78499220,22cyclictest0-21swapper/223:03:382
78499220,21cyclictest3650-21ssh21:46:242
78499220,21cyclictest3650-21ssh21:46:242
78499220,21cyclictest14098-21ssh21:55:342
78499220,21cyclictest13600-21ssh23:50:332
78499220,1cyclictest0-21swapper/223:35:072
782992222,0cyclictest0-21swapper/123:30:501
782992222,0cyclictest0-21swapper/122:40:111
782992220,1cyclictest5531-21cron19:20:001
782992219,2cyclictest22813-21awk19:55:001
78299221,1cyclictest0-21swapper/100:06:501
78299221,1cyclictest0-21swapper/100:06:501
78299220,21cyclictest6623-21sh23:15:391
78299220,21cyclictest3488-21sh00:10:151
78299220,21cyclictest23450-21apt-get19:59:501
78299220,1cyclictest31049-21apt-get20:14:501
78299220,0cyclictest0-21swapper/123:35:481
781992221,1cyclictest0-21swapper/021:47:350
781992221,1cyclictest0-21swapper/021:47:350
781992221,0cyclictest0-21swapper/020:44:490
781992220,1cyclictest20458-21awk20:55:000
781992220,1cyclictest0-21swapper/022:00:040
781992220,1cyclictest0-21swapper/019:20:000
781992220,1cyclictest0-21swapper/000:15:190
78199220,22cyclictest0-21swapper/022:56:220
78199220,21cyclictest0-21swapper/019:14:500
78199220,0cyclictest0-21swapper/021:51:250
785992120,1cyclictest8011-21idleruntime-cro19:25:013
785992120,0cyclictest0-21swapper/319:39:513
785992119,1cyclictest0-21swapper/320:40:003
78599210,21cyclictest0-21swapper/322:27:383
78599210,20cyclictest1033-21apt-get20:19:503
78599210,1cyclictest18656-21apt-get19:49:503
78599210,0cyclictest11058-21apt-get19:34:513
784992121,0cyclictest0-21swapper/222:30:352
784992121,0cyclictest0-21swapper/222:30:352
784992120,0cyclictest18696-21apt-get20:54:482
784992119,1cyclictest17853-21awk20:50:002
78499210,21cyclictest0-21swapper/223:31:012
78499210,20cyclictest0-21swapper/220:29:532
78499210,20cyclictest0-21swapper/219:29:492
78499210,0cyclictest21054-21apt-get19:54:502
78499210,0cyclictest0-21swapper/222:15:372
78499210,0cyclictest0-21swapper/220:09:492
78299210,0cyclictest0-21swapper/120:19:501
78299210,0cyclictest0-21swapper/119:49:501
781992119,1cyclictest0-21swapper/022:20:380
78199210,21cyclictest0-21swapper/020:59:480
78199210,20cyclictest0-21swapper/019:45:000
78199210,0cyclictest0-21swapper/020:49:520
78199210,0cyclictest0-21swapper/019:34:510
785992019,0cyclictest25900-21apt-get21:09:493
784992020,0cyclictest0-21swapper/221:04:512
784992020,0cyclictest0-21swapper/220:59:482
784992019,1cyclictest6209-21apt-get19:24:502
784992018,1cyclictest12666-21munin-run20:40:002
784992018,1cyclictest10073-21idleruntime-cro20:35:002
78499200,20cyclictest0-21swapper/221:09:502
78499200,0cyclictest0-21swapper/219:14:502
782992020,0cyclictest0-21swapper/121:04:511
782992020,0cyclictest0-21swapper/120:35:021
782992020,0cyclictest0-21swapper/119:14:511
782992019,1cyclictest0-21swapper/119:34:511
782992019,0cyclictest0-21swapper/119:24:491
782992018,1cyclictest5213-21ls20:25:011
78299200,0cyclictest0-21swapper/120:44:481
78299200,0cyclictest0-21swapper/119:39:511
781992020,0cyclictest0-21swapper/021:09:490
781992020,0cyclictest0-21swapper/019:54:500
781992020,0cyclictest0-21swapper/019:49:500
78199200,0cyclictest0-21swapper/020:09:490
785991919,0cyclictest0-21swapper/320:44:483
785991918,1cyclictest5568-21perf19:20:003
785991917,1cyclictest22776-21users19:55:003
784991919,0cyclictest13510-21apt-get20:44:492
78499190,0cyclictest0-21swapper/220:04:512
781991919,0cyclictest0-21swapper/019:39:510
785991818,0cyclictest0-21swapper/321:04:513
785991818,0cyclictest0-21swapper/319:14:513
785991817,1cyclictest8308-21apt-get20:34:503
785991817,1cyclictest21088-21apt-get20:59:483
782991818,0cyclictest0-21swapper/120:59:481
781991818,0cyclictest0-21swapper/020:29:530
781991818,0cyclictest0-21swapper/020:19:500
781991818,0cyclictest0-21swapper/020:10:020
781991818,0cyclictest0-21swapper/019:24:500
784991717,0cyclictest0-21swapper/220:10:022
784991717,0cyclictest0-21swapper/219:49:502
784991714,2cyclictest23418-21/usr/sbin/munin20:00:002
784991616,0cyclictest0-21swapper/219:19:502
784991615,1cyclictest16261-21apt-get19:44:502
78499160,0cyclictest0-21swapper/219:34:512
782991616,0cyclictest0-21swapper/121:09:491
78299160,1cyclictest0-21swapper/120:10:011
78199160,1cyclictest25281-21munin-run21:05:010
78199160,16cyclictest0-21swapper/019:29:490
78199160,15cyclictest27476-21cron20:05:000
78199160,0cyclictest0-21swapper/019:59:500
785991515,0cyclictest0-21swapper/319:59:513
785991513,1cyclictest10425-21perf19:30:013
78599150,15cyclictest30012-21apt-get20:10:013
785991414,0cyclictest0-21swapper/320:02:033
78599140,14cyclictest0-21swapper/320:46:053
78499140,14cyclictest0-21swapper/220:20:162
78499140,14cyclictest0-21swapper/220:17:312
782991414,0cyclictest0-21swapper/120:00:481
78299140,0cyclictest0-21swapper/120:46:151
78299140,0cyclictest0-21swapper/119:25:161
78199140,14cyclictest0-21swapper/020:33:150
78199140,14cyclictest0-21swapper/020:20:580
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional