You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-07 - 01:40

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Thu Aug 06, 2026 12:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3209699260,25cyclictest4103-21ssh10:16:392
32090992623,2cyclictest25377-21/usr/sbin/munin11:04:070
3209099260,25cyclictest2747-21date10:15:010
3209099260,25cyclictest0-21swapper/011:14:060
32101992523,1cyclictest669-21ssh11:38:563
32096992521,1cyclictest26277-21diskmemload12:33:362
32090992522,2cyclictest0-21swapper/012:36:550
32090992520,4cyclictest0-21swapper/012:14:040
3209099250,24cyclictest11620-21idleruntime-cro07:35:000
3209099250,24cyclictest0-21swapper/012:15:340
3209099250,24cyclictest0-21swapper/010:03:030
3209099250,24cyclictest0-21swapper/009:44:280
32101992423,1cyclictest26277-21diskmemload11:10:093
3210199242,21cyclictest26277-21diskmemload10:15:103
32101992421,2cyclictest31959-21apt08:19:033
32101992421,2cyclictest21767-21ssh11:56:373
3210199240,24cyclictest0-21swapper/312:29:503
3210199240,24cyclictest0-21swapper/312:29:503
3210199240,23cyclictest25108-21taskset10:34:193
3210199240,23cyclictest17562-21sed11:24:133
32096992423,1cyclictest26277-21diskmemload12:09:252
32096992421,2cyclictest26277-21diskmemload09:50:012
3209699240,24cyclictest0-21swapper/209:28:592
3209699240,23cyclictest28902-21ssh09:41:162
3209699240,23cyclictest0-21swapper/212:36:332
3209699240,23cyclictest0-21swapper/210:47:362
3209699240,1cyclictest19661-21apt-get10:59:042
3209699240,1cyclictest0-21swapper/209:35:382
32094992424,0cyclictest26277-21diskmemload09:28:091
32094992423,1cyclictest15787-21apt-get09:59:041
32094992422,1cyclictest1896-21ssh09:45:581
32094992422,1cyclictest11609-21cron07:35:001
3209499240,24cyclictest0-21swapper/109:13:101
3209499240,23cyclictest18593-21ssh10:00:041
3209499240,23cyclictest0-21swapper/110:19:071
3209499240,0cyclictest26277-21diskmemload11:15:071
3209499240,0cyclictest0-21swapper/110:11:061
32090992424,0cyclictest0-21swapper/011:37:330
32090992423,1cyclictest31546-21ssh12:04:500
3209099240,24cyclictest26277-21diskmemload12:09:520
3209099240,24cyclictest0-21swapper/010:26:140
3209099240,23cyclictest29609-21sh12:32:050
3209099240,23cyclictest0-21swapper/011:50:300
3209099240,23cyclictest0-21swapper/010:59:060
3209099240,23cyclictest0-21swapper/009:26:300
3209099240,22cyclictest26277-21diskmemload12:21:540
3209099240,0cyclictest26277-21diskmemload11:40:220
32101992323,0cyclictest26277-21diskmemload12:03:323
32101992323,0cyclictest26277-21diskmemload09:10:113
32101992323,0cyclictest0-21swapper/312:05:193
32101992323,0cyclictest0-21swapper/311:27:173
32101992323,0cyclictest0-21swapper/310:40:463
32101992323,0cyclictest0-21swapper/310:05:543
32101992323,0cyclictest0-21swapper/309:55:483
32101992323,0cyclictest0-21swapper/309:50:313
32101992321,1cyclictest0-21swapper/311:46:193
32101992321,1cyclictest0-21swapper/311:40:143
32101992321,1cyclictest0-21swapper/311:30:503
32101992321,1cyclictest0-21swapper/310:53:513
32101992321,1cyclictest0-21swapper/310:21:473
3210199231,0cyclictest6193-21apt-get08:30:023
3210199230,23cyclictest26277-21diskmemload11:02:183
3210199230,23cyclictest0-21swapper/312:33:413
3210199230,23cyclictest0-21swapper/312:17:473
3210199230,23cyclictest0-21swapper/310:55:063
3210199230,23cyclictest0-21swapper/309:34:473
3210199230,22cyclictest31386-21awk08:15:003
3210199230,22cyclictest26277-21diskmemload12:20:203
3210199230,22cyclictest26277-21diskmemload10:25:453
3210199230,22cyclictest0-21swapper/309:44:073
3210199230,22cyclictest0-21swapper/307:59:033
3210199230,22cyclictest0-21swapper/307:54:033
3210199230,0cyclictest26277-21diskmemload09:35:133
3210199230,0cyclictest0-21swapper/309:49:043
32096992323,0cyclictest26277-21diskmemload09:56:482
32096992323,0cyclictest0-21swapper/211:49:272
32096992323,0cyclictest0-21swapper/209:16:062
32096992323,0cyclictest0-21swapper/209:16:062
32096992322,0cyclictest0-21swapper/210:03:192
32096992321,1cyclictest26277-21diskmemload10:08:132
32096992321,1cyclictest20934-21ssh11:55:292
32096992321,1cyclictest0-21swapper/209:10:142
3209699230,22cyclictest16702-21sh10:27:482
3209699230,22cyclictest13938-21apt-get10:54:022
3209699230,22cyclictest0-21swapper/211:00:252
3209699230,22cyclictest0-21swapper/209:20:582
3209699230,0cyclictest0-21swapper/212:29:462
3209699230,0cyclictest0-21swapper/212:29:462
3209699230,0cyclictest0-21swapper/211:26:242
3209699230,0cyclictest0-21swapper/211:10:572
3209699230,0cyclictest0-21swapper/210:39:342
32094992323,0cyclictest26277-21diskmemload11:31:111
32094992323,0cyclictest26277-21diskmemload10:32:351
32094992323,0cyclictest0-21swapper/112:25:441
32094992323,0cyclictest0-21swapper/112:25:441
32094992323,0cyclictest0-21swapper/112:14:191
32094992323,0cyclictest0-21swapper/111:23:311
32094992323,0cyclictest0-21swapper/110:45:171
32094992323,0cyclictest0-21swapper/110:37:001
32094992323,0cyclictest0-21swapper/110:25:161
32094992323,0cyclictest0-21swapper/109:15:181
32094992323,0cyclictest0-21swapper/109:15:181
32094992321,1cyclictest15877-21sh11:51:181
32094992320,2cyclictest9374-21sh11:45:131
3209499230,23cyclictest26277-21diskmemload10:56:531
3209499230,23cyclictest0-21swapper/112:37:031
3209499230,23cyclictest0-21swapper/110:23:091
3209499230,22cyclictest8784-21ssh09:24:041
3209499230,22cyclictest4275-21rm11:40:591
3209499230,22cyclictest30414-21sh12:33:121
3209499230,22cyclictest26277-21diskmemload11:39:331
3209499230,22cyclictest25203-21ssh10:06:361
3209499230,22cyclictest21688-21sh11:56:241
3209499230,22cyclictest18605-21sh09:32:421
3209499230,1cyclictest0-21swapper/112:00:251
3209499230,0cyclictest0-21swapper/109:44:311
32090992323,0cyclictest0-21swapper/011:07:110
32090992323,0cyclictest0-21swapper/010:20:020
32090992320,2cyclictest10857-21ssh11:47:120
32090992320,2cyclictest0-21swapper/009:34:030
3209099230,23cyclictest0-21swapper/011:21:000
3209099230,23cyclictest0-21swapper/009:48:220
3209099230,22cyclictest7928-21sh11:16:170
3209099230,22cyclictest30725-21chrt10:39:190
3209099230,22cyclictest27729-21perf09:40:000
3209099230,22cyclictest26277-21diskmemload10:42:050
3209099230,22cyclictest21556-21ssh10:31:500
3209099230,22cyclictest14203-21latency_hist08:45:010
3209099230,21cyclictest0-21swapper/011:55:090
3209099230,0cyclictest26277-21diskmemload10:52:000
3209099230,0cyclictest26277-21diskmemload09:51:040
32101992222,0cyclictest0-21swapper/311:16:263
32101992222,0cyclictest0-21swapper/309:20:173
3210199220,21cyclictest606-21dbus-daemon08:10:003
3210199220,21cyclictest4364-21apt-get08:29:023
3210199220,21cyclictest32610-21apt-get07:14:013
3210199220,21cyclictest28033-21ssh11:05:023
3210199220,21cyclictest0-21swapper/310:45:513
3210199220,21cyclictest0-21swapper/308:40:023
3210199220,1cyclictest7409-21apt-get07:29:033
3210199220,1cyclictest5806-21ssh12:10:503
3210199220,1cyclictest32680-21apt-get10:14:033
3210199220,1cyclictest24381-21apt-get08:04:033
3210199220,1cyclictest18960-21sh10:00:423
3210199220,1cyclictest1610-21diskmemload12:35:193
3210199220,1cyclictest0-21swapper/311:50:443
3210199220,1cyclictest0-21swapper/310:35:053
32096992222,0cyclictest0-21swapper/212:16:132
32096992222,0cyclictest0-21swapper/212:01:332
32096992222,0cyclictest0-21swapper/209:32:082
32096992221,1cyclictest21966-21apt-get07:59:042
32096992220,1cyclictest16594-21dump-pmu-power07:45:002
32096992220,1cyclictest11410-21sendmail-msp08:40:012
32096992220,1cyclictest0-21swapper/211:30:502
32096992220,1cyclictest0-21swapper/207:49:032
3209699221,1cyclictest0-21swapper/208:25:002
3209699220,22cyclictest26277-21diskmemload11:17:252
3209699220,22cyclictest0-21swapper/212:11:322
3209699220,22cyclictest0-21swapper/211:05:112
3209699220,21cyclictest17160-21ssh12:20:422
3209699220,21cyclictest0-21swapper/210:40:022
3209699220,21cyclictest0-21swapper/210:10:192
3209699220,21cyclictest0-21swapper/208:15:002
3209699220,1cyclictest0-21swapper/211:21:242
3209699220,1cyclictest0-21swapper/210:30:412
3209699220,1cyclictest0-21swapper/210:20:152
3209699220,0cyclictest26277-21diskmemload09:46:512
32094992222,0cyclictest0-21swapper/112:15:031
32094992222,0cyclictest0-21swapper/111:00:491
32094992222,0cyclictest0-21swapper/109:35:221
32094992221,1cyclictest0-21swapper/110:52:431
32094992221,0cyclictest26277-21diskmemload10:41:041
32094992220,1cyclictest11407-21idleruntime-cro08:40:011
3209499220,22cyclictest7430-21ssh09:50:201
3209499220,22cyclictest0-21swapper/111:05:181
3209499220,21cyclictest313-21systemd-journal08:15:001
3209499220,21cyclictest26277-21diskmemload11:25:321
32090992221,1cyclictest0-21swapper/009:20:210
32090992221,0cyclictest0-21swapper/007:29:030
32090992220,1cyclictest16616-21munin-run07:45:000
32090992220,1cyclictest14125-21turbostat07:40:000
32090992220,1cyclictest0-21swapper/011:30:010
3209099221,21cyclictest62550irq/127-eno110:15:020
3209099220,22cyclictest14426-21ssh09:57:180
3209099220,21cyclictest27491-21rm09:11:290
3209099220,21cyclictest26277-21diskmemload12:25:180
3209099220,21cyclictest26277-21diskmemload12:25:180
3209099220,21cyclictest26277-21diskmemload11:25:060
3209099220,21cyclictest0-21swapper/008:05:020
3209099220,21cyclictest0-21swapper/008:00:010
3209099220,1cyclictest11408-21latency_hist08:40:010
32101992121,0cyclictest14818-21apt-get08:49:023
32101992120,1cyclictest0-21swapper/307:49:033
32101992119,1cyclictest4411-21sendmail07:20:003
3210199210,1cyclictest14143-21munin-run07:40:003
3210199210,0cyclictest0-21swapper/309:17:303
3210199210,0cyclictest0-21swapper/309:17:303
32096992121,0cyclictest0-21swapper/211:40:192
32096992121,0cyclictest0-21swapper/208:44:032
32096992121,0cyclictest0-21swapper/208:30:022
32096992121,0cyclictest0-21swapper/207:54:032
32096992119,1cyclictest26200-21apt-get08:05:022
3209699210,20cyclictest0-21swapper/207:14:012
3209699210,1cyclictest6195-21sh08:30:002
32094992119,1cyclictest14204-21grep08:45:011
32094992118,2cyclictest1368-21cron08:20:001
3209499210,21cyclictest0-21swapper/107:54:031
3209499210,20cyclictest0-21swapper/108:59:021
3209499210,20cyclictest0-21swapper/107:29:031
32090992120,0cyclictest22028-21apt-get09:04:020
32090992120,0cyclictest0-21swapper/008:14:030
3209099210,1cyclictest1971-21apt-get08:24:020
3209099210,0cyclictest0-21swapper/008:04:020
3209099210,0cyclictest0-21swapper/007:19:010
3210199200,20cyclictest0-21swapper/308:24:023
32096992020,0cyclictest0-21swapper/208:54:022
32096992019,0cyclictest19610-21apt-get08:59:022
32096992019,0cyclictest0-21swapper/209:09:032
32094992020,0cyclictest0-21swapper/108:24:021
32094992020,0cyclictest0-21swapper/108:05:021
32094992020,0cyclictest0-21swapper/108:04:031
32094992017,2cyclictest14134-21cron07:40:001
32090992020,0cyclictest0-21swapper/007:14:010
32090992018,1cyclictest6832-21cron07:25:000
32101991918,1cyclictest0-21swapper/309:09:033
32101991917,1cyclictest341ksoftirqd/308:40:013
32096991919,0cyclictest0-21swapper/207:29:032
32096991919,0cyclictest0-21swapper/207:24:022
32096991917,1cyclictest4451-21sendmail07:20:002
32094991919,0cyclictest32610-21apt-get07:14:011
32094991919,0cyclictest0-21swapper/107:19:011
3209499190,0cyclictest0-21swapper/109:04:021
3209499190,0cyclictest0-21swapper/108:49:021
3209499190,0cyclictest0-21swapper/107:24:021
32090991919,0cyclictest0-21swapper/007:49:020
32101991815,2cyclictest21459-21perf09:00:003
32094991818,0cyclictest0-21swapper/108:35:011
32101991717,0cyclictest5014-21apt-get07:24:023
32096991716,1cyclictest9805-21apt-get07:34:012
32090991717,0cyclictest0-21swapper/008:30:020
32096991615,1cyclictest31960-21apt-get08:19:022
32090991616,0cyclictest0-21swapper/009:09:030
32090991616,0cyclictest0-21swapper/008:49:020
32101991515,0cyclictest0-21swapper/307:44:023
3210199150,1cyclictest0-21swapper/307:34:103
32096991515,0cyclictest0-21swapper/208:49:022
32090991513,1cyclictest21471-21idleruntime-cro09:00:000
3210199140,14cyclictest0-21swapper/309:00:053
3210199140,14cyclictest0-21swapper/308:50:333
3209699140,14cyclictest0-21swapper/209:00:272
3209699140,14cyclictest0-21swapper/208:00:452
3209699140,14cyclictest0-21swapper/207:35:552
32094991414,0cyclictest0-21swapper/109:07:131
3209499140,14cyclictest0-21swapper/108:25:481
32090991414,0cyclictest0-21swapper/008:15:040
3209099140,14cyclictest0-21swapper/008:50:100
3209099140,0cyclictest0-21swapper/008:25:010
3209099140,0cyclictest0-21swapper/007:51:050
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional