You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-17 - 17:29

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Wed Jun 17, 2026 12:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
442599260,25cyclictest6061-21ssh10:18:093
4420992624,1cyclictest0-21swapper/110:18:091
4425992522,2cyclictest22265-21sh09:32:463
442599250,24cyclictest30350-21diskmemload10:41:563
4422992523,1cyclictest7630-21apt-get09:19:572
4422992523,1cyclictest22149-21apt-get08:54:552
442299250,22cyclictest27884-21rm12:36:372
4415992523,1cyclictest0-21swapper/010:00:000
4415992522,2cyclictest15480-21sed11:25:050
4425992422,1cyclictest0-21swapper/311:08:583
442599240,2cyclictest5363-21diskmemload11:46:033
442599240,24cyclictest30350-21diskmemload12:17:333
442599240,23cyclictest8565-21munin-run08:25:003
442599240,23cyclictest0-21swapper/312:24:303
442599240,23cyclictest0-21swapper/312:13:263
442599240,23cyclictest0-21swapper/310:34:193
442599240,23cyclictest0-21swapper/310:27:153
442599240,22cyclictest0-21swapper/311:40:013
442599240,1cyclictest0-21swapper/309:26:003
442599240,0cyclictest0-21swapper/312:39:403
4422992422,1cyclictest0-21swapper/212:15:002
4422992422,1cyclictest0-21swapper/210:04:342
4422992421,2cyclictest28-21ksoftirqd/211:40:012
442299240,23cyclictest30350-21diskmemload10:40:242
442299240,23cyclictest25246-21df12:34:592
442299240,23cyclictest20202-21date12:00:002
442299240,23cyclictest15754-21dump-pmu-power08:40:012
442299240,23cyclictest0-21swapper/209:46:402
442299240,1cyclictest30350-21diskmemload09:58:572
442299240,1cyclictest21151-21rm11:00:412
442299240,1cyclictest0-21swapper/209:28:372
4420992421,2cyclictest578-21rm10:42:471
4420992421,2cyclictest22836-21ssh10:03:521
442099240,24cyclictest0-21swapper/111:01:571
442099240,23cyclictest7424-21ssh11:49:091
442099240,23cyclictest0-21swapper/109:26:001
442099240,23cyclictest0-21swapper/109:18:201
4415992423,1cyclictest62550irq/127-eno109:50:190
4415992423,1cyclictest0-21swapper/011:07:280
4415992421,2cyclictest4853-21ssh10:16:230
441599240,2cyclictest14222-21awk09:25:010
441599240,23cyclictest30816-21idleruntime-cro11:40:010
441599240,1cyclictest8947-21munin-run11:50:010
441599240,0cyclictest0-21swapper/010:12:220
4425992323,0cyclictest30350-21diskmemload11:22:323
4425992323,0cyclictest30350-21diskmemload10:08:083
4425992323,0cyclictest0-21swapper/310:01:013
4425992323,0cyclictest0-21swapper/309:54:483
4425992323,0cyclictest0-21swapper/309:39:073
4425992323,0cyclictest0-21swapper/309:10:103
4425992322,1cyclictest0-21swapper/312:02:143
4425992322,1cyclictest0-21swapper/311:26:463
4425992322,0cyclictest0-21swapper/312:26:303
4425992321,1cyclictest0-21swapper/309:46:523
4425992320,2cyclictest22081-21ssh11:02:103
442599231,21cyclictest14347-21apt-get11:55:013
442599230,2cyclictest2156-21sh11:44:203
442599230,23cyclictest0-21swapper/310:36:393
442599230,23cyclictest0-21swapper/310:20:263
442599230,23cyclictest0-21swapper/310:10:373
442599230,23cyclictest0-21swapper/309:44:333
442599230,22cyclictest7559-21ssh11:19:043
442599230,22cyclictest14278-21smartctl09:25:013
442599230,22cyclictest0-21swapper/311:10:033
442599230,22cyclictest0-21swapper/310:48:463
442599230,1cyclictest20506-21apt-get07:44:563
442599230,0cyclictest0-21swapper/311:57:143
442599230,0cyclictest0-21swapper/310:59:593
4422992323,0cyclictest0-21swapper/211:46:072
4422992323,0cyclictest0-21swapper/209:53:552
4422992321,1cyclictest0-21swapper/210:37:282
4422992321,1cyclictest0-21swapper/210:19:562
4422992321,1cyclictest0-21swapper/210:12:162
4422992318,4cyclictest0-21swapper/209:10:412
4422992318,1cyclictest23851-21apt-get09:34:572
442299230,23cyclictest30350-21diskmemload10:09:012
442299230,23cyclictest0-21swapper/212:16:342
442299230,23cyclictest0-21swapper/211:43:042
442299230,23cyclictest0-21swapper/211:31:022
442299230,23cyclictest0-21swapper/210:59:002
442299230,23cyclictest0-21swapper/210:30:362
442299230,22cyclictest8420-21cron08:25:002
442299230,22cyclictest28794-21cron08:00:012
442299230,22cyclictest21490-21munin-run07:45:002
442299230,22cyclictest0-21swapper/210:50:162
442299230,0cyclictest0-21swapper/209:24:392
4420992323,0cyclictest30350-21diskmemload12:13:261
4420992323,0cyclictest0-21swapper/112:17:331
4420992323,0cyclictest0-21swapper/111:57:141
4420992323,0cyclictest0-21swapper/110:59:441
4420992323,0cyclictest0-21swapper/110:36:391
4420992323,0cyclictest0-21swapper/110:26:341
4420992323,0cyclictest0-21swapper/110:20:261
4420992323,0cyclictest0-21swapper/109:49:211
4420992323,0cyclictest0-21swapper/109:44:331
4420992323,0cyclictest0-21swapper/109:37:381
4420992321,1cyclictest31118-21ssh09:10:051
4420992321,1cyclictest0-21swapper/112:02:141
4420992320,2cyclictest10475-21ssh11:50:461
442099230,2cyclictest26793-21ssh10:06:401
442099230,23cyclictest0-21swapper/110:11:071
442099230,23cyclictest0-21swapper/109:57:331
442099230,22cyclictest7407-21ssh11:18:521
442099230,22cyclictest29243-21ssh12:08:201
442099230,22cyclictest29107-21ssh11:09:301
442099230,22cyclictest28234-21sh12:37:161
442099230,22cyclictest1542-21ssh11:13:171
442099230,22cyclictest11582-21ssh12:21:171
442099230,22cyclictest11143-21ssh11:21:281
442099230,0cyclictest0-21swapper/112:27:461
442099230,0cyclictest0-21swapper/110:32:171
4415992323,0cyclictest0-21swapper/009:41:250
4415992322,1cyclictest62550irq/127-eno111:19:170
4415992322,1cyclictest30350-21diskmemload12:28:310
4415992322,1cyclictest0-21swapper/009:26:240
4415992321,2cyclictest0-21swapper/012:05:440
4415992321,1cyclictest0-21swapper/011:43:420
4415992321,1cyclictest0-21swapper/011:11:040
4415992321,1cyclictest0-21swapper/010:51:020
4415992320,2cyclictest27434-21ssh10:37:300
441599231,22cyclictest62550irq/127-eno112:04:240
441599230,2cyclictest30061-21ssh12:39:560
441599230,23cyclictest30350-21diskmemload10:34:140
441599230,23cyclictest0-21swapper/011:02:500
441599230,23cyclictest0-21swapper/010:40:320
441599230,23cyclictest0-21swapper/010:07:150
441599230,23cyclictest0-21swapper/009:31:310
441599230,22cyclictest6513-21ssh10:48:340
441599230,22cyclictest21481-21missed_timers07:45:000
441599230,22cyclictest18817-21diskstats10:29:580
441599230,22cyclictest15602-21ssh10:55:270
441599230,22cyclictest12522-21ssh10:24:150
441599230,22cyclictest11314-21sh11:51:560
441599230,0cyclictest0-21swapper/012:31:230
4425992222,0cyclictest0-21swapper/310:50:063
4425992220,1cyclictest32556-21apt-get08:09:573
4425992220,1cyclictest0-21swapper/309:09:563
4425992220,1cyclictest0-21swapper/307:20:013
442599220,2cyclictest4567-21sh09:15:203
442599220,21cyclictest30350-21diskmemload12:07:493
442599220,21cyclictest0-21swapper/312:30:463
442599220,21cyclictest0-21swapper/309:55:413
442599220,21cyclictest0-21swapper/308:29:573
442599220,1cyclictest11659-21cron07:25:003
4422992222,0cyclictest0-21swapper/211:07:162
4422992222,0cyclictest0-21swapper/207:50:012
4422992220,1cyclictest9978-21timerwakeupswit11:50:062
4422992220,1cyclictest774-21runrttasks12:04:362
4422992219,2cyclictest10615-21sh11:20:392
442299220,21cyclictest4416-21ssh10:45:252
442299220,21cyclictest30350-21diskmemload09:40:482
442299220,21cyclictest21745-21kworker/u8:211:11:272
442299220,21cyclictest18054-21sh12:27:582
442299220,21cyclictest10789-21apt-get07:24:562
442299220,21cyclictest0-21swapper/208:14:562
442299220,1cyclictest17738-21ssh11:28:142
442299220,1cyclictest0-21swapper/212:22:052
442299220,1cyclictest0-21swapper/211:15:462
442299220,1cyclictest0-21swapper/209:39:312
4420992222,0cyclictest0-21swapper/111:40:181
4420992222,0cyclictest0-21swapper/109:31:171
4420992220,1cyclictest9741-21sh10:50:231
4420992219,2cyclictest9850-21ssh09:50:251
442099220,21cyclictest30350-21diskmemload09:20:581
442099220,21cyclictest21033-21sh11:30:101
442099220,21cyclictest11680-21cron07:25:001
442099220,21cyclictest0-21swapper/110:48:071
442099220,21cyclictest0-21swapper/108:34:551
442099220,21cyclictest0-21swapper/108:19:571
442099220,0cyclictest0-21swapper/107:44:561
4415992222,0cyclictest30350-21diskmemload09:36:080
4415992220,2cyclictest10711-21ssh12:20:070
4415992219,2cyclictest11708-21basename07:25:000
441599220,21cyclictest27936-21latency_hist09:05:000
441599220,21cyclictest15977-21ssh11:55:160
441599220,21cyclictest0-21swapper/011:30:060
441599220,21cyclictest0-21swapper/010:01:470
441599220,21cyclictest0-21swapper/008:59:570
441599220,21cyclictest0-21swapper/008:54:550
441599220,1cyclictest5724-21ssh09:47:220
441599220,1cyclictest30350-21diskmemload11:22:310
441599220,1cyclictest12546-21apt-get08:34:550
4422992120,1cyclictest7077-21turbostat07:15:012
4422992120,1cyclictest0-21swapper/207:34:572
442299210,1cyclictest19752-21apt-get08:49:562
442299210,0cyclictest0-21swapper/209:09:562
4420992121,0cyclictest0-21swapper/108:29:571
4420992119,1cyclictest9433-21ntp_states07:20:011
4415992121,0cyclictest22950-21apt-get07:49:550
4415992121,0cyclictest0-21swapper/008:44:590
4415992121,0cyclictest0-21swapper/008:39:560
4415992119,1cyclictest28851-21sendmail-msp08:00:010
441599210,21cyclictest30350-21diskmemload09:15:220
441599210,0cyclictest0-21swapper/008:19:570
4425992020,0cyclictest0-21swapper/308:19:573
4425992019,0cyclictest0-21swapper/307:49:553
442599200,19cyclictest3591-21/usr/sbin/munin08:15:003
442599200,0cyclictest0-21swapper/307:14:553
4422992020,0cyclictest0-21swapper/209:04:562
4422992019,0cyclictest0-21swapper/207:29:572
4422992019,0cyclictest0-21swapper/207:20:012
442299200,20cyclictest0-21swapper/208:09:572
4420992020,0cyclictest0-21swapper/107:14:551
4420992019,1cyclictest29409-21apt-get09:09:561
4420992019,1cyclictest0-21swapper/108:09:571
442099200,0cyclictest17363-21apt-get08:44:591
4415992020,0cyclictest0-21swapper/008:49:560
4415992020,0cyclictest0-21swapper/007:54:560
4415992020,0cyclictest0-21swapper/007:54:560
4415992020,0cyclictest0-21swapper/007:19:550
441599200,0cyclictest18078-21apt-get07:39:560
4425991919,0cyclictest0-21swapper/308:54:553
4425991919,0cyclictest0-21swapper/307:54:563
4425991919,0cyclictest0-21swapper/307:54:563
4425991918,1cyclictest25347-21perf09:00:003
4422991919,0cyclictest12546-21apt-get08:34:552
442299190,18cyclictest30955-21munin-run08:05:012
4415991919,0cyclictest15585-21apt-get07:34:570
4415991919,0cyclictest0-21swapper/007:14:550
441599190,18cyclictest30131-21/usr/sbin/munin08:05:030
441599190,0cyclictest0-21swapper/008:04:580
4425991818,0cyclictest0-21swapper/307:59:553
4425991818,0cyclictest0-21swapper/307:39:553
4422991818,0cyclictest25347-21apt-get07:54:552
4422991818,0cyclictest25347-21apt-get07:54:552
4422991817,1cyclictest10991-21perf08:30:002
442299180,17cyclictest6125-21cron08:20:002
4420991818,0cyclictest0-21swapper/108:54:551
4420991818,0cyclictest0-21swapper/107:54:561
4420991818,0cyclictest0-21swapper/107:54:561
4420991816,1cyclictest25371-21munin-run09:00:001
442099180,18cyclictest3597-21latency_hist08:15:001
4425991716,1cyclictest0-21swapper/307:29:573
4425991716,0cyclictest0-21swapper/308:34:553
4420991717,0cyclictest0-21swapper/108:24:581
4420991717,0cyclictest0-21swapper/107:59:551
4420991717,0cyclictest0-21swapper/107:49:551
4420991717,0cyclictest0-21swapper/107:39:551
4420991715,1cyclictest14010-21perf07:30:001
4420991714,2cyclictest31015-21sort08:05:011
4415991717,0cyclictest0-21swapper/007:29:560
4415991716,0cyclictest10149-21apt-get08:29:570
4415991714,2cyclictest3596-21apt-key08:15:000
4425991616,0cyclictest0-21swapper/308:49:563
4425991616,0cyclictest0-21swapper/308:39:573
4425991616,0cyclictest0-21swapper/307:34:573
4425991614,1cyclictest0-21swapper/309:00:023
4420991615,0cyclictest0-21swapper/108:39:571
4415991616,0cyclictest0-21swapper/008:25:000
442599150,8cyclictest0-21swapper/308:40:033
4422991515,0cyclictest0-21swapper/208:44:592
442299150,11cyclictest1-21systemd09:00:002
4420991515,0cyclictest0-21swapper/107:34:571
442099150,11cyclictest611-21irqbalance08:49:221
442599140,14cyclictest0-21swapper/308:01:183
442299140,14cyclictest0-21swapper/207:35:132
4420991414,0cyclictest0-21swapper/109:00:251
4415991414,0cyclictest0-21swapper/009:05:040
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional