You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-14 - 23:46

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot5s.osadl.org (updated Thu May 14, 2026 12:44:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3313992519,5cyclictest15287-21sh12:16:112
3309992523,1cyclictest0-21swapper/109:11:281
3309992522,2cyclictest11566-21pmu-power10:47:121
330899250,24cyclictest19614-21ssh10:54:450
330899250,1cyclictest29881-21diskmemload11:08:310
330899250,1cyclictest18864-21sh12:18:500
3318992423,1cyclictest29881-21diskmemload11:58:153
3318992423,1cyclictest22087-21ssh11:25:293
331899240,2cyclictest30046-21rm09:37:563
331899240,24cyclictest29881-21diskmemload11:32:563
331899240,24cyclictest0-21swapper/310:52:163
331899240,23cyclictest5157-21ssh11:38:343
331899240,1cyclictest29881-21diskmemload10:06:113
3313992422,1cyclictest0-21swapper/211:41:372
3313992422,1cyclictest0-21swapper/210:05:012
331399240,23cyclictest29881-21diskmemload11:24:222
331399240,23cyclictest0-21swapper/209:47:052
331399240,1cyclictest13367-21ssh10:20:442
331399240,1cyclictest0-21swapper/211:07:062
3309992424,0cyclictest0-21swapper/111:58:121
3309992422,1cyclictest313-21systemd-journal07:50:011
3309992422,1cyclictest0-21swapper/111:02:011
330999240,23cyclictest2958-21/usr/sbin/munin10:12:071
330999240,23cyclictest2958-21/usr/sbin/munin10:12:071
330999240,23cyclictest0-21swapper/112:38:471
330999240,23cyclictest0-21swapper/110:25:481
330999240,23cyclictest0-21swapper/109:22:131
330999240,23cyclictest0-21swapper/109:22:131
330999240,0cyclictest0-21swapper/111:25:291
330999240,0cyclictest0-21swapper/109:50:001
330999240,0cyclictest0-21swapper/109:41:091
3308992423,1cyclictest29881-21diskmemload11:18:220
3308992423,1cyclictest0-21swapper/011:25:030
3308992420,3cyclictest0-21swapper/012:12:140
330899241,22cyclictest10639-21basename08:30:010
330899240,24cyclictest0-21swapper/009:36:240
330899240,23cyclictest8683-21apt-get10:17:070
330899240,23cyclictest0-21swapper/010:57:070
330899240,0cyclictest29881-21diskmemload10:09:240
330899240,0cyclictest29881-21diskmemload09:19:190
330899240,0cyclictest0-21swapper/010:37:310
3318992323,0cyclictest29881-21diskmemload12:35:553
3318992323,0cyclictest29881-21diskmemload12:05:203
3318992323,0cyclictest29881-21diskmemload09:20:503
3318992323,0cyclictest29881-21diskmemload09:20:503
3318992323,0cyclictest0-21swapper/312:30:073
3318992323,0cyclictest0-21swapper/311:42:143
3318992323,0cyclictest0-21swapper/311:22:253
3318992323,0cyclictest0-21swapper/310:32:153
3318992323,0cyclictest0-21swapper/310:10:053
3318992323,0cyclictest0-21swapper/310:10:053
3318992323,0cyclictest0-21swapper/309:41:253
3318992322,1cyclictest29881-21diskmemload12:22:123
3318992321,1cyclictest725-21sh12:02:203
331899230,23cyclictest29881-21diskmemload09:49:473
331899230,23cyclictest0-21swapper/310:48:513
331899230,23cyclictest0-21swapper/309:50:213
331899230,22cyclictest22491-21sh10:00:073
331899230,22cyclictest21657-21sh09:31:373
331899230,22cyclictest18853-21ssh11:51:203
331899230,22cyclictest14152-21ssh10:21:533
331899230,0cyclictest29881-21diskmemload12:26:373
331899230,0cyclictest29881-21diskmemload11:01:163
331899230,0cyclictest29881-21diskmemload09:16:313
331899230,0cyclictest0-21swapper/310:43:263
331899230,0cyclictest0-21swapper/310:26:503
3313992323,0cyclictest0-21swapper/212:25:032
3313992323,0cyclictest0-21swapper/212:11:272
3313992323,0cyclictest0-21swapper/211:49:042
3313992323,0cyclictest0-21swapper/211:16:252
3313992323,0cyclictest0-21swapper/209:40:112
3313992321,1cyclictest0-21swapper/212:36:302
3313992321,1cyclictest0-21swapper/210:15:062
3313992321,1cyclictest0-21swapper/207:50:012
3313992320,2cyclictest6860-21cron10:15:012
3313992320,2cyclictest6860-21cron10:15:012
3313992320,2cyclictest30102-21rm09:10:012
331399230,2cyclictest25024-21ssh10:59:202
331399230,23cyclictest0-21swapper/211:12:442
331399230,22cyclictest7513-21ssh12:08:452
331399230,22cyclictest5833-21idleruntime-cro08:20:002
331399230,22cyclictest29881-21diskmemload09:50:152
331399230,22cyclictest24330-21ssh10:30:202
331399230,22cyclictest0-21swapper/210:39:402
331399230,22cyclictest0-21swapper/209:55:212
331399230,1cyclictest0-21swapper/212:32:082
331399230,0cyclictest0-21swapper/212:24:332
331399230,0cyclictest0-21swapper/212:01:222
331399230,0cyclictest0-21swapper/210:27:582
331399230,0cyclictest0-21swapper/209:23:052
331399230,0cyclictest0-21swapper/209:23:052
3309992323,0cyclictest0-21swapper/111:15:431
3309992323,0cyclictest0-21swapper/109:38:251
3309992322,0cyclictest0-21swapper/112:10:211
3309992322,0cyclictest0-21swapper/112:05:141
3309992321,1cyclictest0-21swapper/110:40:201
3309992321,1cyclictest0-21swapper/109:32:131
3309992320,2cyclictest24419-21ssh10:30:281
3309992320,1cyclictest0-21swapper/110:04:201
330999230,23cyclictest11720-21ssh11:44:481
330999230,23cyclictest0-21swapper/110:52:161
330999230,23cyclictest0-21swapper/110:07:391
330999230,22cyclictest25815-21apt-get07:57:051
330999230,22cyclictest23386-21dump-pmu-power11:55:011
330999230,22cyclictest17540-21cron11:50:011
330999230,22cyclictest0-21swapper/108:47:061
330999230,0cyclictest0-21swapper/112:33:091
330999230,0cyclictest0-21swapper/111:36:351
330999230,0cyclictest0-21swapper/110:58:551
330999230,0cyclictest0-21swapper/109:55:091
3308992323,0cyclictest29881-21diskmemload09:42:180
3308992323,0cyclictest0-21swapper/011:45:060
3308992323,0cyclictest0-21swapper/011:34:260
3308992323,0cyclictest0-21swapper/009:32:400
3308992321,2cyclictest24768-21sh11:56:090
3308992321,1cyclictest7141-21sh11:41:000
3308992321,1cyclictest30443-21ssh12:00:580
3308992321,1cyclictest0-21swapper/010:32:580
3308992320,2cyclictest2550-21ssh12:32:220
330899231,1cyclictest62550irq/127-eno109:11:040
330899230,23cyclictest0-21swapper/009:29:060
330899230,22cyclictest5226-21ssh11:10:400
330899230,22cyclictest27822-21lockfile-create08:00:010
330899230,22cyclictest20320-21ssh12:20:260
330899230,22cyclictest1247-21rm11:35:410
330899230,22cyclictest0-21swapper/012:35:270
330899230,22cyclictest0-21swapper/009:45:550
330899230,22cyclictest0-21swapper/009:20:300
330899230,22cyclictest0-21swapper/009:20:300
330899230,0cyclictest0-21swapper/012:26:020
330899230,0cyclictest0-21swapper/010:44:420
3318992222,0cyclictest0-21swapper/312:10:133
3318992222,0cyclictest0-21swapper/311:11:173
3318992222,0cyclictest0-21swapper/309:25:113
3318992220,2cyclictest29881-21diskmemload09:10:183
3318992220,1cyclictest22427-21sed10:00:013
3318992220,1cyclictest0-21swapper/308:35:013
331899220,22cyclictest29881-21diskmemload12:15:093
331899220,21cyclictest5828-21perf08:20:003
331899220,21cyclictest10594-21ssh11:15:153
331899220,21cyclictest0-21swapper/311:45:133
331899220,21cyclictest0-21swapper/311:05:043
331899220,21cyclictest0-21swapper/310:35:203
331899220,21cyclictest0-21swapper/307:42:043
331899220,21cyclictest0-21swapper/307:42:043
331899220,1cyclictest10882-21rm10:17:423
3313992222,0cyclictest29881-21diskmemload10:51:272
3313992222,0cyclictest0-21swapper/210:46:252
3313992222,0cyclictest0-21swapper/209:36:532
3313992221,0cyclictest6270-21apt-get08:22:072
3313992220,1cyclictest15685-21awk07:35:002
3313992220,1cyclictest0-21swapper/211:25:472
331399221,1cyclictest28963-21ssh11:32:012
331399220,21cyclictest20371-21munin-run09:30:012
331399220,21cyclictest0-21swapper/211:00:212
331399220,21cyclictest0-21swapper/210:40:582
331399220,21cyclictest0-21swapper/210:06:142
331399220,21cyclictest0-21swapper/209:15:192
331399220,21cyclictest0-21swapper/209:10:062
331399220,21cyclictest0-21swapper/208:07:052
331399220,21cyclictest0-21swapper/207:40:012
331399220,1cyclictest21274-21ssh09:31:082
331399220,1cyclictest0-21swapper/211:35:572
3309992222,0cyclictest0-21swapper/109:29:171
3309992221,1cyclictest0-21swapper/111:05:261
3309992221,0cyclictest0-21swapper/112:27:411
3309992220,2cyclictest15703-21rm12:16:521
3309992220,1cyclictest0-21swapper/110:35:011
330999220,21cyclictest774-21runrttasks10:20:031
330999220,21cyclictest29881-21diskmemload09:15:131
330999220,21cyclictest0-21swapper/110:15:101
330999220,21cyclictest0-21swapper/109:50:271
330999220,21cyclictest0-21swapper/108:52:071
330999220,1cyclictest13038-21sh08:35:011
330999220,1cyclictest0-21swapper/112:20:351
330999220,1cyclictest0-21swapper/111:32:071
330999220,1cyclictest0-21swapper/111:22:001
330999220,1cyclictest0-21swapper/111:12:151
3308992222,0cyclictest0-21swapper/010:00:120
3308992220,1cyclictest8245-21latency_hist08:25:000
3308992220,1cyclictest10488-21apt-get07:25:000
3308992220,1cyclictest0-21swapper/011:50:050
330899221,21cyclictest13-21ksoftirqd/009:56:170
330899220,21cyclictest9740-21rm10:46:230
330899220,21cyclictest13457-21apt-get08:37:070
330899220,1cyclictest0-21swapper/012:07:070
330899220,1cyclictest0-21swapper/011:01:260
330899220,0cyclictest0-21swapper/010:22:440
3318992121,0cyclictest0-21swapper/308:47:063
3318992121,0cyclictest0-21swapper/307:22:063
3318992119,1cyclictest22991-21apt07:50:013
331899210,21cyclictest30647-21apt-get08:05:023
331899210,20cyclictest13282-21cron07:30:003
331899210,0cyclictest20664-21apt-get08:52:063
331899210,0cyclictest0-21swapper/310:58:003
3313992120,1cyclictest28682-21apt-get08:02:062
3313992119,1cyclictest23421-21apt-get07:52:062
331399210,1cyclictest0-21swapper/208:47:052
331399210,0cyclictest11064-21apt-get08:32:072
3309992120,0cyclictest0-21swapper/108:42:051
330999210,21cyclictest0-21swapper/109:07:071
330999210,0cyclictest8526-21apt-get07:22:061
3308992121,0cyclictest29881-21diskmemload09:50:180
3308992121,0cyclictest0-21swapper/008:02:060
3308992120,1cyclictest0-21swapper/011:20:030
3308992119,1cyclictest10488-21apt-get07:25:020
330899210,0cyclictest0-21swapper/009:07:070
330899210,0cyclictest0-21swapper/007:37:060
330899210,0cyclictest0-21swapper/007:12:060
3318992020,0cyclictest15874-21apt-get08:42:063
3318992020,0cyclictest0-21swapper/307:57:053
3318992019,1cyclictest27872-21apt-get09:07:073
331899200,0cyclictest1436-21apt-get08:12:053
331899200,0cyclictest0-21swapper/308:22:073
3313992020,0cyclictest8661-21apt-get08:27:052
3313992020,0cyclictest0-21swapper/208:57:072
3313992020,0cyclictest0-21swapper/207:42:042
3313992020,0cyclictest0-21swapper/207:42:042
3313992018,1cyclictest15444-21turbostat08:40:012
331399200,20cyclictest0-21swapper/207:27:052
3309992020,0cyclictest0-21swapper/108:37:071
3309992020,0cyclictest0-21swapper/108:22:071
3309992020,0cyclictest0-21swapper/107:25:021
3308992020,0cyclictest0-21swapper/008:52:070
3308992019,1cyclictest18628-21apt-get07:42:050
3308992019,1cyclictest18628-21apt-get07:42:050
330899200,19cyclictest0-21swapper/007:52:060
330899200,0cyclictest25475-21apt-get09:02:070
3318991919,0cyclictest0-21swapper/308:27:053
3309991917,1cyclictest15706-21munin-run07:35:001
3309991917,1cyclictest0-21swapper/107:15:001
330999190,18cyclictest0-21swapper/108:57:081
3308991919,0cyclictest18269-21apt-get08:47:050
3308991919,0cyclictest0-21swapper/008:17:050
3308991919,0cyclictest0-21swapper/007:47:050
3318991817,1cyclictest0-21swapper/308:37:073
3313991818,0cyclictest0-21swapper/208:12:052
3309991817,1cyclictest30655-21perf08:05:001
3308991818,0cyclictest0-21swapper/008:57:070
3308991818,0cyclictest0-21swapper/007:17:050
3318991716,1cyclictest23061-21apt-get08:57:083
331899170,16cyclictest3712-21apt-get07:12:063
3309991717,0cyclictest0-21swapper/109:02:071
3309991717,0cyclictest0-21swapper/108:12:051
3309991717,0cyclictest0-21swapper/108:07:051
3309991716,0cyclictest0-21swapper/108:27:051
3309991715,1cyclictest18225-21grep07:40:011
330899170,1cyclictest3409-21cron08:15:000
3318991614,1cyclictest30647-21apt-get08:05:003
3318991614,1cyclictest0-21swapper/307:35:003
3313991611,4cyclictest607-21in:imuxsock08:55:002
3309991616,0cyclictest0-21swapper/108:17:051
3309991616,0cyclictest0-21swapper/107:52:061
330999160,15cyclictest0-21swapper/107:42:041
330999160,15cyclictest0-21swapper/107:42:041
3308991616,0cyclictest0-21swapper/008:42:060
3308991615,1cyclictest11064-21apt-get08:32:060
3313991515,0cyclictest0-21swapper/207:17:052
331399150,1cyclictest0-21swapper/207:57:512
3309991513,1cyclictest0-21swapper/107:17:131
331899140,14cyclictest0-21swapper/309:00:043
331899140,14cyclictest0-21swapper/307:50:443
331899140,0cyclictest0-21swapper/307:35:323
331899140,0cyclictest0-21swapper/307:15:033
3313991414,0cyclictest0-21swapper/207:20:302
331399140,14cyclictest0-21swapper/209:00:182
331399140,14cyclictest0-21swapper/208:40:402
331399140,14cyclictest0-21swapper/207:11:342
330899140,14cyclictest0-21swapper/007:30:030
330899140,0cyclictest0-21swapper/008:05:210
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional