You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-03 - 20:17

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot5s.osadl.org (updated Fri Apr 03, 2026 12:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
30257992625,1cyclictest28-21ksoftirqd/208:45:012
3025199260,25cyclictest23588-21diskmemload09:14:241
3026199250,24cyclictest0-21swapper/309:58:453
30257992520,4cyclictest23588-21diskmemload10:07:392
3025799250,1cyclictest13112-21ssh09:30:312
30251992525,0cyclictest0-21swapper/111:44:151
3025199250,24cyclictest12146-21apt-get08:48:391
3024699252,1cyclictest62550irq/127-eno112:24:570
3024699250,24cyclictest6792-21apt-get10:28:410
30261992424,0cyclictest0-21swapper/311:35:513
3026199240,23cyclictest8182-21ssh11:00:213
3026199240,23cyclictest23588-21diskmemload10:05:153
3026199240,23cyclictest22136-21kworker/u8:010:25:323
3026199240,23cyclictest12662-21apt-get12:08:413
3026199240,23cyclictest0-21swapper/311:11:083
3026199240,23cyclictest0-21swapper/310:45:303
3026199240,23cyclictest0-21swapper/310:31:483
30257992422,1cyclictest0-21swapper/211:18:432
3025799240,23cyclictest1938-21sendmail-msp09:20:012
3025799240,23cyclictest0-21swapper/212:33:162
3025799240,23cyclictest0-21swapper/210:56:542
30251992423,1cyclictest27600-21ssh09:44:231
3025199240,23cyclictest23588-21diskmemload12:31:311
3025199240,23cyclictest23588-21diskmemload11:01:241
3025199240,23cyclictest0-21swapper/112:13:161
3025199240,23cyclictest0-21swapper/110:45:331
3025199240,23cyclictest0-21swapper/108:53:401
3024699240,24cyclictest23588-21diskmemload09:52:200
3024699240,24cyclictest0-21swapper/012:00:200
3024699240,24cyclictest0-21swapper/011:26:160
3024699240,23cyclictest21948-21perf08:00:000
3024699240,23cyclictest15100-21ssh11:08:130
3024699240,23cyclictest0-21swapper/010:31:420
3024699240,0cyclictest23588-21diskmemload09:15:250
30261992323,0cyclictest23588-21diskmemload09:20:043
30261992323,0cyclictest0-21swapper/310:51:203
30261992323,0cyclictest0-21swapper/309:38:083
30261992322,1cyclictest23588-21diskmemload11:05:423
30261992321,1cyclictest0-21swapper/311:15:123
30261992321,1cyclictest0-21swapper/310:36:433
30261992321,1cyclictest0-21swapper/309:19:233
3026199230,23cyclictest23588-21diskmemload12:27:153
3026199230,23cyclictest23588-21diskmemload09:25:133
3026199230,23cyclictest0-21swapper/312:02:213
3026199230,23cyclictest0-21swapper/311:30:123
3026199230,23cyclictest0-21swapper/310:01:573
3026199230,23cyclictest0-21swapper/309:50:443
3026199230,22cyclictest9557-21chrt12:36:253
3026199230,22cyclictest313-21systemd-journal11:50:013
3026199230,22cyclictest30512-21ssh09:48:403
3026199230,22cyclictest23588-21diskmemload12:31:013
3026199230,22cyclictest23588-21diskmemload12:10:123
3026199230,22cyclictest0-21swapper/310:10:043
3026199230,1cyclictest23588-21diskmemload11:25:153
3026199230,0cyclictest23588-21diskmemload11:55:303
3026199230,0cyclictest0-21swapper/312:24:253
30257992323,0cyclictest0-21swapper/211:42:412
30257992323,0cyclictest0-21swapper/210:12:272
30257992323,0cyclictest0-21swapper/209:29:362
30257992321,1cyclictest24528-21sh09:42:142
30257992321,1cyclictest0-21swapper/212:18:562
30257992320,3cyclictest0-21swapper/210:47:232
30257992320,2cyclictest9634-21sh12:04:222
30257992320,2cyclictest30757-21ssh10:50:582
30257992320,2cyclictest23588-21diskmemload09:49:282
3025799230,23cyclictest0-21swapper/212:24:212
3025799230,23cyclictest0-21swapper/211:46:062
3025799230,23cyclictest0-21swapper/211:14:552
3025799230,23cyclictest0-21swapper/210:25:382
3025799230,23cyclictest0-21swapper/209:10:382
3025799230,22cyclictest2581-21ssh09:20:472
3025799230,22cyclictest23588-21diskmemload10:15:452
3025799230,22cyclictest18907-21ssh12:13:472
3025799230,22cyclictest10989-21sh12:06:082
3025799230,22cyclictest0-21swapper/211:50:162
3025799230,1cyclictest0-21swapper/211:32:492
3025799230,0cyclictest0-21swapper/210:43:012
3025799230,0cyclictest0-21swapper/209:56:152
30251992323,0cyclictest0-21swapper/111:17:111
30251992323,0cyclictest0-21swapper/109:22:241
30251992321,1cyclictest7663-21apt-get09:25:021
30251992321,1cyclictest0-21swapper/110:08:431
30251992320,2cyclictest1785-21sh09:51:201
30251992320,2cyclictest10985-21idleruntime-cro10:00:011
3025199230,2cyclictest2597-21ssh11:26:331
3025199230,2cyclictest15051-21sh11:08:081
3025199230,23cyclictest0-21swapper/112:29:521
3025199230,23cyclictest0-21swapper/112:09:571
3025199230,23cyclictest0-21swapper/111:45:551
3025199230,23cyclictest0-21swapper/109:36:151
3025199230,22cyclictest8636-21ssh11:33:011
3025199230,22cyclictest32590-21latency_hist07:15:001
3025199230,22cyclictest30811-21ssh09:17:211
3025199230,22cyclictest22653-21ssh10:11:471
3025199230,22cyclictest0-21swapper/110:42:441
3025199230,0cyclictest0-21swapper/110:59:571
3025199230,0cyclictest0-21swapper/110:36:391
3024699233,0cyclictest62550irq/127-eno112:13:410
30246992323,0cyclictest23588-21diskmemload12:25:260
30246992323,0cyclictest23588-21diskmemload11:53:350
30246992323,0cyclictest23588-21diskmemload10:57:140
30246992323,0cyclictest23588-21diskmemload09:39:100
30246992323,0cyclictest0-21swapper/011:23:420
30246992321,1cyclictest15456-21ssh10:03:580
30246992321,1cyclictest0-21swapper/011:17:400
30246992321,1cyclictest0-21swapper/009:55:090
3024699231,21cyclictest62550irq/127-eno109:45:140
3024699230,23cyclictest0-21swapper/011:41:480
3024699230,22cyclictest25068-21ssh10:45:050
3024699230,22cyclictest23588-21diskmemload11:31:280
3024699230,22cyclictest23588-21diskmemload11:00:300
3024699230,22cyclictest23588-21diskmemload10:53:080
3024699230,22cyclictest22852-21ssh11:46:040
3024699230,22cyclictest19542-21turbostat.cron07:55:010
3024699230,22cyclictest12338-21ssh11:35:450
3024699230,22cyclictest0-21swapper/012:16:010
3024699230,1cyclictest0-21swapper/012:36:380
3024699230,0cyclictest23588-21diskmemload12:06:370
3024699230,0cyclictest0-21swapper/010:40:420
3024699230,0cyclictest0-21swapper/010:05:360
3024699230,0cyclictest0-21swapper/009:31:580
3024699230,0cyclictest0-21swapper/009:10:040
30261992220,1cyclictest17692-21apt-get07:53:393
3026199221,0cyclictest0-21swapper/310:15:403
3026199220,21cyclictest29691-21apt-get08:18:393
3026199220,21cyclictest23588-21diskmemload11:20:403
3026199220,21cyclictest2098-21apt-get08:28:393
3026199220,21cyclictest20781-21ssh12:15:213
3026199220,21cyclictest20098-21sh10:40:233
3026199220,21cyclictest13128-21rm09:30:333
3026199220,21cyclictest0-21swapper/311:50:043
3026199220,21cyclictest0-21swapper/310:20:383
3026199220,21cyclictest0-21swapper/309:45:003
3026199220,21cyclictest0-21swapper/309:03:403
3026199220,21cyclictest0-21swapper/307:33:403
3026199220,1cyclictest0-21swapper/311:43:363
3026199220,1cyclictest0-21swapper/310:56:283
3026199220,0cyclictest23588-21diskmemload09:10:143
30257992222,0cyclictest0-21swapper/211:20:262
30257992222,0cyclictest0-21swapper/211:06:312
30257992220,1cyclictest18408-21ssh09:35:472
30257992220,1cyclictest0-21swapper/212:26:182
30257992219,2cyclictest971-21sh09:50:012
3025799220,2cyclictest8775-21sendmail-msp08:40:012
3025799220,2cyclictest8120-21ssh11:00:162
3025799220,21cyclictest901-21ssh11:56:282
3025799220,21cyclictest31873-21ssh10:20:112
3025799220,21cyclictest20085-21apt-get07:58:392
3025799220,21cyclictest0-21swapper/211:35:332
3025799220,21cyclictest0-21swapper/211:25:552
3025799220,21cyclictest0-21swapper/210:32:412
3025799220,1cyclictest0-21swapper/210:00:022
3025799220,1cyclictest0-21swapper/208:18:402
30251992222,0cyclictest23588-21diskmemload09:46:401
30251992222,0cyclictest0-21swapper/111:36:141
30251992222,0cyclictest0-21swapper/110:51:181
30251992222,0cyclictest0-21swapper/109:34:011
30251992222,0cyclictest0-21swapper/108:33:391
30251992220,1cyclictest12726-21ssh10:02:231
30251992219,2cyclictest12441-21fschecks_time10:33:411
3025199220,22cyclictest0-21swapper/111:13:491
3025199220,22cyclictest0-21swapper/110:15:211
3025199220,21cyclictest0-21swapper/112:15:481
3025199220,21cyclictest0-21swapper/110:29:101
3025199220,21cyclictest0-21swapper/110:20:321
3025199220,1cyclictest5534-21apt-get07:28:391
3025199220,1cyclictest30906-21ssh11:23:121
3025199220,1cyclictest0-21swapper/112:00:471
3025199220,1cyclictest0-21swapper/111:51:431
3025199220,0cyclictest0-21swapper/111:55:131
3025199220,0cyclictest0-21swapper/108:28:391
30246992222,0cyclictest32501-21ssh10:21:070
30246992222,0cyclictest0-21swapper/012:30:130
30246992222,0cyclictest0-21swapper/010:13:140
30246992220,1cyclictest0-21swapper/009:40:010
3024699220,22cyclictest8658-21ssh09:26:400
3024699220,21cyclictest31583-21basename08:20:000
3024699220,21cyclictest24347-21awk08:05:000
3024699220,21cyclictest15616-21ssh10:36:190
3024699220,21cyclictest10323-21apt07:38:390
3024699220,21cyclictest0-21swapper/011:55:190
3024699220,21cyclictest0-21swapper/011:11:080
3024699220,21cyclictest0-21swapper/007:45:010
3024699220,0cyclictest23588-21diskmemload10:15:060
30261992120,1cyclictest24896-21apt-get08:08:383
3026199210,20cyclictest619-21polkitd07:40:013
30257992121,0cyclictest0-21swapper/209:08:402
30257992120,1cyclictest16942-21apt-get08:58:402
30257992119,1cyclictest11155-21apt-get08:45:022
3025799210,1cyclictest2592-21awk07:20:002
30251992121,0cyclictest0-21swapper/107:48:391
30251992120,1cyclictest25913-21ssh12:20:181
30251992120,1cyclictest22505-21apt-get08:03:401
30251992119,1cyclictest21927-21cron08:00:001
30246992119,1cyclictest1527-21cron08:25:010
3024699210,20cyclictest0-21swapper/008:53:400
3024699210,0cyclictest21753-21apt-get09:08:400
3024699210,0cyclictest0-21swapper/008:33:400
30261992020,0cyclictest0-21swapper/308:58:403
30261992018,1cyclictest11155-21apt-get08:45:013
3026199200,20cyclictest0-21swapper/307:28:393
3026199200,0cyclictest0-21swapper/308:33:393
30257992020,0cyclictest0-21swapper/209:03:402
30257992020,0cyclictest0-21swapper/208:03:402
30257992020,0cyclictest0-21swapper/207:33:402
30251992020,0cyclictest0-21swapper/108:18:391
30251992020,0cyclictest0-21swapper/107:33:401
30251992018,1cyclictest19556-21sleep07:55:011
30246992020,0cyclictest0-21swapper/008:08:380
30246992020,0cyclictest0-21swapper/007:13:390
30261991919,0cyclictest0-21swapper/308:48:393
30261991919,0cyclictest0-21swapper/308:03:403
30257991918,1cyclictest14546-21apt-get08:53:402
30257991918,1cyclictest0-21swapper/207:43:392
30251991919,0cyclictest0-21swapper/108:08:391
30251991919,0cyclictest0-21swapper/107:18:381
30251991918,1cyclictest27290-21apt-get08:13:391
30251991917,1cyclictest8736-21sh08:40:011
30246991919,0cyclictest0-21swapper/008:43:400
30246991919,0cyclictest0-21swapper/007:25:000
30261991818,0cyclictest695-21apt-get07:18:383
30257991818,0cyclictest0-21swapper/208:13:392
30257991818,0cyclictest0-21swapper/207:53:392
30257991818,0cyclictest0-21swapper/207:48:392
30251991818,0cyclictest0-21swapper/109:08:401
30251991816,1cyclictest4992-21latency_hist07:25:001
30246991818,0cyclictest0-21swapper/008:38:380
30246991816,1cyclictest313-21systemd-journal08:15:000
30261991717,0cyclictest0-21swapper/308:53:403
30261991717,0cyclictest0-21swapper/308:23:393
30257991715,1cyclictest0-21swapper/207:25:002
30251991717,0cyclictest9307-21apt-get08:43:401
30251991717,0cyclictest10324-21apt-get07:38:391
30251991715,1cyclictest19360-21apt-get09:03:401
30246991717,0cyclictest0-21swapper/007:48:390
3024699170,0cyclictest0-21swapper/007:28:390
30261991616,0cyclictest0-21swapper/307:23:383
30261991614,1cyclictest0-21swapper/307:43:443
30261991614,1cyclictest0-21swapper/307:15:003
3025799160,14cyclictest0-21swapper/208:30:212
3025199160,15cyclictest0-21swapper/108:58:401
30246991616,0cyclictest0-21swapper/008:48:390
30246991614,1cyclictest9776-21dump-pmu-power07:35:010
30246991614,1cyclictest2424-21grep07:18:480
30246991614,1cyclictest0-21swapper/008:56:400
3024699160,15cyclictest2621-21perl08:28:410
30261991515,0cyclictest0-21swapper/309:08:403
30257991515,0cyclictest0-21swapper/208:08:382
3026199140,14cyclictest0-21swapper/308:35:053
3026199140,14cyclictest0-21swapper/308:10:113
3026199140,14cyclictest0-21swapper/307:55:263
3026199140,0cyclictest0-21swapper/307:47:413
30257991414,0cyclictest0-21swapper/207:25:182
3025799140,14cyclictest0-21swapper/208:25:062
3025799140,14cyclictest0-21swapper/208:21:302
3025799140,14cyclictest0-21swapper/207:11:072
3025799140,0cyclictest0-21swapper/207:35:172
30251991414,0cyclictest0-21swapper/108:20:101
3025199140,0cyclictest0-21swapper/107:40:091
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional