You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-05 - 14:41

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Wed Aug 05, 2026 12:44:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
31742204166,25sleep00-21swapper/007:08:140
32242203167,24sleep30-21swapper/307:08:533
29582202165,25sleep10-21swapper/107:05:211
29442202167,23sleep20-21swapper/207:05:102
1890621410,2sleep2340699cyclictest10:50:242
340699106102,3cyclictest0-21swapper/211:34:282
3406996865,2cyclictest0-21swapper/208:01:552
312862560,1sleep20-21swapper/210:05:212
293392540,3sleep20-21swapper/212:30:012
12872520,1sleep00-21swapper/012:39:450
301532460,1sleep10-21swapper/112:30:161
3394993912,13cyclictest11330-21munin-run08:10:010
339499389,13cyclictest22217-21munin-run09:40:020
3406993733,3cyclictest0-21swapper/212:33:072
339499378,13cyclictest23435-21munin-run09:50:000
339499367,13cyclictest10876-21idleruntime-cro10:35:010
339499321,11cyclictest10682-21kworker/0:008:20:470
340199301,23cyclictest0-21swapper/110:01:551
340199300,5cyclictest0-21swapper/111:20:221
340199300,5cyclictest0-21swapper/111:20:221
339499305,12cyclictest25632-21munin-run09:55:000
3394993011,16cyclictest9285-21munin-run11:45:010
339499295,11cyclictest392-21basename11:25:010
339499295,11cyclictest392-21basename11:25:010
340199271,18cyclictest0-21swapper/110:10:421
340199271,12cyclictest0-21swapper/109:16:091
340199271,11cyclictest21365-21hddtemp_smartct12:10:161
340699261,4cyclictest0-21swapper/211:56:222
3406992614,7cyclictest23057-40dnf09:45:252
340199262,11cyclictest13141-21munin-run08:25:021
340199262,10cyclictest0-21swapper/109:14:271
340199261,11cyclictest21281-21timerwakeupswit10:55:251
340199260,4cyclictest31918-21kworker/u8:010:38:421
339499262,11cyclictest14349-21munin-run08:35:010
339499261,4cyclictest0-21swapper/010:16:120
339499261,12cyclictest13758-21idleruntime-cro08:30:020
98592251,20sleep310064-21perl11:45:153
341199251,17cyclictest0-21swapper/310:45:213
340199251,4cyclictest0-21swapper/112:25:061
340199251,3cyclictest0-21swapper/109:28:041
340199250,6cyclictest0-21swapper/112:05:361
340199250,6cyclictest0-21swapper/111:13:231
339499252,16cyclictest6096-21kworker/0:207:36:010
341199241,8cyclictest0-21swapper/311:18:203
340699247,8cyclictest87350irq/126-enp1s0-10:10:232
340699244,8cyclictest5672-21perl11:35:162
340699241,6cyclictest0-21swapper/211:07:062
340199249,3cyclictest11486-21munin-run11:50:011
340199241,6cyclictest0-21swapper/112:35:031
340199241,12cyclictest0-21swapper/109:33:211
340199241,11cyclictest2640-21diskstats10:15:141
340199241,11cyclictest24925-21awk09:50:201
340199241,11cyclictest23663-21users11:00:261
340199240,6cyclictest0-21swapper/111:34:171
340199240,5cyclictest0-21swapper/111:37:571
340199240,5cyclictest0-21swapper/111:27:091
340199240,5cyclictest0-21swapper/109:35:181
341199235,4cyclictest18584-21diskmemload09:45:243
340699238,7cyclictest87350irq/126-enp1s0-10:25:192
340699236,9cyclictest12527-21sh08:20:002
340699235,9cyclictest13736-21latency_hist08:30:012
340699234,6cyclictest24927-21awk12:20:012
340699231,7cyclictest0-21swapper/211:50:132
3406992310,5cyclictest87350irq/126-enp1s0-11:40:212
340199231,6cyclictest0-21swapper/111:59:031
340199231,3cyclictest14346-21perf08:35:011
340199231,2cyclictest0-21swapper/110:23:191
340199231,19cyclictest25490-21diskstats12:20:141
340199231,16cyclictest0-21swapper/111:07:361
340199231,10cyclictest0-21swapper/110:32:221
340199230,6cyclictest0-21swapper/111:16:461
340199230,14cyclictest0-21swapper/109:45:241
339499231,3cyclictest0-21swapper/009:31:270
143482230,1sleep30-21swapper/308:35:013
341199226,6cyclictest0-21swapper/311:27:093
341199225,10cyclictest0-21swapper/310:22:383
341199221,9cyclictest87350irq/126-enp1s0-09:15:013
341199221,6cyclictest0-21swapper/312:30:583
341199221,5cyclictest25131-21kworker/u8:012:24:183
341199221,4cyclictest0-21swapper/312:30:023
341199221,3cyclictest0-21swapper/312:35:433
341199221,16cyclictest0-21swapper/311:05:283
340699221,8cyclictest311ktimersoftd/209:35:022
340699221,4cyclictest0-21swapper/210:03:062
340699221,12cyclictest0-21swapper/212:35:182
340699220,11cyclictest10573-21ntp_states11:45:222
340199221,10cyclictest18583-21latency_hist09:10:011
3401992210,2cyclictest0-21swapper/109:55:131
339499226,8cyclictest18925-21wc10:50:250
339499221,3cyclictest0-21swapper/012:08:480
341199216,5cyclictest0-21swapper/311:58:073
341199213,9cyclictest0-21swapper/311:35:153
341199213,3cyclictest0-21swapper/310:38:313
341199211,5cyclictest0-21swapper/309:57:183
341199211,5cyclictest0-21swapper/309:29:053
341199211,4cyclictest0-21swapper/310:07:003
341199211,4cyclictest0-21swapper/310:00:583
3411992114,4cyclictest15549-40dnf08:45:013
3411992114,3cyclictest0-21swapper/310:55:573
341199210,7cyclictest0-21swapper/311:11:123
340699216,7cyclictest7685-21latency_hist07:40:012
340699216,7cyclictest30874-21/usr/sbin/munin11:20:242
340699216,7cyclictest30874-21/usr/sbin/munin11:20:242
340699215,3cyclictest18598-21perf09:10:012
340699213,8cyclictest12050irq/123-ahci[0009:00:012
340699213,2cyclictest618-21/usr/sbin/munin11:25:222
340699211,5cyclictest0-21swapper/209:27:032
340699211,4cyclictest0-21swapper/210:15:212
340699211,12cyclictest0-21swapper/209:35:152
340199217,4cyclictest0-21swapper/110:54:381
340199211,9cyclictest0-21swapper/108:09:371
3401992110,5cyclictest0-21swapper/112:15:301
340199210,5cyclictest0-21swapper/109:21:251
339499214,7cyclictest0-21swapper/012:25:000
339499211,6cyclictest0-21swapper/011:48:030
341199206,4cyclictest0-21swapper/309:41:473
341199205,5cyclictest0-21swapper/309:19:543
341199203,2cyclictest87350irq/126-enp1s0-11:40:133
341199201,6cyclictest0-21swapper/310:30:283
341199201,6cyclictest0-21swapper/310:16:233
341199201,4cyclictest0-21swapper/309:32:213
3411992014,3cyclictest24060-21ls11:05:013
341199201,3cyclictest0-21swapper/311:32:373
341199201,11cyclictest11792-21/usr/sbin/munin11:50:233
341199201,11cyclictest0-21swapper/312:12:293
341199201,11cyclictest0-21swapper/310:53:113
341199201,11cyclictest0-21swapper/310:40:193
341199200,9cyclictest18070-21ps12:02:453
340699206,2cyclictest11926-21latency_hist08:15:012
340699205,8cyclictest27542-21ntp_states11:10:212
340699202,8cyclictest87550irq/128-enp1s0-08:53:142
340699202,10cyclictest311ktimersoftd/207:15:022
340699201,9cyclictest0-21swapper/212:10:042
340699201,5cyclictest0-21swapper/210:46:282
340699201,11cyclictest0-21swapper/212:00:192
3401992016,3cyclictest0-21swapper/111:40:181
3401992016,3cyclictest0-21swapper/110:44:311
340199201,2cyclictest14627-21runrttasks08:35:561
340199201,18cyclictest0-21swapper/111:50:071
340199201,17cyclictest0-21swapper/108:25:531
340199201,17cyclictest0-21swapper/108:17:581
340199201,17cyclictest0-21swapper/107:51:291
340199201,17cyclictest0-21swapper/107:51:281
340199201,17cyclictest0-21swapper/107:45:311
340199201,17cyclictest0-21swapper/107:42:201
340199201,10cyclictest0-21swapper/109:40:261
340199200,18cyclictest0-21swapper/110:06:361
339499208,3cyclictest827-21runrttasks11:39:530
339499205,8cyclictest28052-21/usr/sbin/munin10:00:150
339499205,4cyclictest7608-21missed_timers10:25:190
339499201,3cyclictest0-21swapper/011:13:100
339499201,13cyclictest87350irq/126-enp1s0-10:10:230
3394992011,2cyclictest8564-1kworker/u9:112:25:560
339499201,11cyclictest29536-21/usr/sbin/munin12:30:230
339499201,10cyclictest24010-21ntp_states12:15:210
341199191,9cyclictest0-21swapper/308:18:183
341199191,9cyclictest0-21swapper/307:36:463
341199191,4cyclictest0-21swapper/310:14:273
3411991914,3cyclictest41-21ksoftirqd/307:35:013
341199191,2cyclictest87350irq/126-enp1s0-09:50:133
341199191,10cyclictest808-21snmpd08:26:503
341199191,10cyclictest0-21swapper/310:29:523
341199191,10cyclictest0-21swapper/309:22:353
341199191,10cyclictest0-21swapper/308:23:183
340699194,2cyclictest4494-21/usr/sbin/munin10:20:262
340699193,8cyclictest8973-21grep10:30:142
340699193,6cyclictest87550irq/128-enp1s0-07:31:402
340699193,2cyclictest87550irq/128-enp1s0-12:08:432
340699191,8cyclictest311ktimersoftd/208:00:012
340699191,8cyclictest12946-21sshd10:38:462
340699191,8cyclictest0-21swapper/210:44:242
340699191,5cyclictest25143-21/usr/sbin/munin12:20:172
340699191,5cyclictest0-21swapper/211:00:362
340699191,11cyclictest0-21swapper/209:53:262
340199193,3cyclictest0-21swapper/108:40:351
340199191,8cyclictest17011-21unixbench_multi10:45:261
340199191,17cyclictest0-21swapper/108:59:491
340199191,17cyclictest0-21swapper/107:39:511
340199191,17cyclictest0-21swapper/107:30:541
340199191,17cyclictest0-21swapper/107:21:211
340199191,17cyclictest0-21swapper/107:10:441
340199191,16cyclictest0-21swapper/108:10:471
340199191,16cyclictest0-21swapper/108:01:221
340199191,10cyclictest16155-21latency_hist08:50:011
340199190,5cyclictest0-21swapper/110:25:581
340199190,17cyclictest0-21swapper/107:29:021
339499197,3cyclictest0-21swapper/010:56:200
339499195,3cyclictest87350irq/126-enp1s0-07:20:020
339499192,14cyclictest19190-21dump-pmu-power09:15:010
339499192,10cyclictest773-21NetworkManager10:45:470
339499191,6cyclictest14274-21head10:40:200
339499191,5cyclictest0-21swapper/011:51:050
339499191,3cyclictest0-21swapper/011:01:100
339499191,2cyclictest0-21swapper/011:34:380
3394991912,4cyclictest1295-21grep11:25:170
339499191,11cyclictest0-21swapper/009:58:360
339499191,10cyclictest0-21swapper/011:56:400
341199189,2cyclictest18584-21diskmemload11:21:123
341199189,2cyclictest18584-21diskmemload11:21:123
341199188,3cyclictest839-21lldpd08:52:253
341199183,12cyclictest11563-21sa108:10:213
341199181,7cyclictest24910-21sh12:20:013
341199181,15cyclictest0-21swapper/308:57:433
341199181,10cyclictest0-21swapper/312:07:493
341199181,10cyclictest0-21swapper/309:36:253
341199180,9cyclictest0-21swapper/308:01:513
340699185,4cyclictest12050irq/123-ahci[0007:50:012
340699184,3cyclictest808-21snmpd09:01:572
340699184,11cyclictest5270-21sh07:20:022
340699183,7cyclictest0-21swapper/211:15:102
340699181,6cyclictest0-21swapper/209:42:512
340699181,5cyclictest0-21swapper/209:16:582
340699181,15cyclictest0-21swapper/208:06:492
340199181,16cyclictest0-21swapper/112:00:441
340199181,16cyclictest0-21swapper/109:01:251
340199180,16cyclictest0-21swapper/108:50:061
340199180,16cyclictest0-21swapper/107:57:271
340199180,16cyclictest0-21swapper/107:15:581
339499187,3cyclictest0-21swapper/008:12:220
339499185,3cyclictest10682-21kworker/0:009:09:070
339499184,7cyclictest808-21snmpd10:07:100
339499182,5cyclictest29019-21df11:15:160
339499181,9cyclictest0-21swapper/009:19:140
339499181,5cyclictest0-21swapper/010:24:540
3394991815,2cyclictest3214-21kworker/0:007:22:340
3394991810,1cyclictest0-21swapper/012:10:250
91942171,2sleep00-21swapper/007:51:040
91942171,2sleep00-21swapper/007:51:040
341199176,5cyclictest774-21avahi-daemon08:36:333
341199171,6cyclictest808-21snmpd09:06:573
341199171,5cyclictest0-21swapper/307:50:323
341199171,5cyclictest0-21swapper/307:50:323
341199171,15cyclictest0-21swapper/307:27:043
341199171,14cyclictest0-21swapper/308:08:183
3411991711,3cyclictest9050-21cut07:50:013
340699176,3cyclictest15568-21awk08:45:002
340699172,2cyclictest87650irq/129-enp1s0-08:31:502
340699171,7cyclictest20881-21netstat10:55:192
340699171,7cyclictest0-21swapper/209:21:562
340699171,6cyclictest0-21swapper/209:55:132
3406991713,2cyclictest18584-21diskmemload09:11:452
340699171,14cyclictest0-21swapper/208:45:522
339499176,3cyclictest12723-21kworker/u8:310:36:310
339499171,5cyclictest0-21swapper/009:26:210
339499171,4cyclictest24449-21cpuspeed_turbos11:05:130
339499171,4cyclictest16946-21basename12:00:190
339499171,4cyclictest0-21swapper/008:46:550
339499171,15cyclictest0-21swapper/009:42:190
341199161,13cyclictest839-21lldpd09:04:553
340699161,6cyclictest0-21swapper/207:26:002
340699160,14cyclictest0-21swapper/208:20:342
340699160,14cyclictest0-21swapper/207:54:242
340699160,14cyclictest0-21swapper/207:54:232
339499161,6cyclictest0-21swapper/009:20:190
339499161,3cyclictest0-21swapper/007:25:480
339499161,2cyclictest0-21swapper/008:50:440
339499161,2cyclictest0-21swapper/007:42:330
3394991612,3cyclictest0-21swapper/008:02:040
3394991612,3cyclictest0-21swapper/007:55:510
3394991612,3cyclictest0-21swapper/007:45:040
339499161,14cyclictest0-21swapper/008:18:380
339499161,14cyclictest0-21swapper/007:30:520
339499160,14cyclictest0-21swapper/008:40:050
172882160,2sleep00-21swapper/008:59:110
341199151,5cyclictest4500-21irqstats07:15:043
3411991512,2cyclictest0-21swapper/307:22:553
3411991511,3cyclictest0-21swapper/307:10:023
3411991510,3cyclictest827-21runrttasks08:49:583
341199150,14cyclictest0-21swapper/307:41:163
340699152,6cyclictest0-21swapper/207:41:402
3406991512,2cyclictest0-21swapper/207:21:082
340699150,14cyclictest0-21swapper/208:38:102
339499151,3cyclictest0-21swapper/007:12:170
339499151,2cyclictest0-21swapper/008:35:470
341199141,11cyclictest808-21snmpd07:56:443
339499140,5cyclictest0-21swapper/009:03:020
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional