You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-14 - 13:08

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack5slot3s.osadl.org (updated Tue Apr 14, 2026 00:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
305912241165,24sleep30-21swapper/319:05:313
307582203167,23sleep20-21swapper/219:07:352
307722202166,24sleep10-21swapper/119:07:471
306082202164,26sleep00-21swapper/019:05:430
864021740,3sleep23104099cyclictest19:30:142
1139021520,2sleep30-21swapper/320:50:143
308272510,2sleep081ktimersoftd/021:35:010
3104099331,19cyclictest0-21swapper/222:35:532
3104099321,21cyclictest0-21swapper/220:44:362
3104099321,12cyclictest28867-21ntp_states23:55:202
3104099311,21cyclictest0-21swapper/200:26:252
3104099302,12cyclictest1333-21perl21:40:162
3104099301,13cyclictest0-21swapper/222:32:252
3104099301,13cyclictest0-21swapper/222:06:492
3104099301,11cyclictest0-21swapper/219:50:592
3102799304,12cyclictest8138-21munin-run19:30:000
3102799304,12cyclictest26969-21awk22:40:010
3104099291,9cyclictest0-21swapper/222:48:502
3104099291,19cyclictest0-21swapper/221:13:522
3104099291,13cyclictest0-21swapper/221:30:562
3104099291,12cyclictest0-21swapper/222:24:532
31036992911,8cyclictest5581-21kworker/u8:121:07:021
3102799291,13cyclictest13144-21munin-run20:55:010
3104099281,12cyclictest0-21swapper/221:16:042
3104099281,12cyclictest0-21swapper/220:11:552
3104099280,10cyclictest0-21swapper/219:58:112
3104099271,12cyclictest0-21swapper/200:37:492
3104099271,12cyclictest0-21swapper/200:34:572
3104099271,10cyclictest0-21swapper/223:05:002
3104099271,10cyclictest0-21swapper/219:35:032
3104099270,12cyclictest0-21swapper/223:15:002
3104099270,12cyclictest0-21swapper/223:15:002
3104099261,8cyclictest0-21swapper/221:35:142
3104099261,13cyclictest0-21swapper/223:49:402
3104099261,13cyclictest0-21swapper/200:20:252
3104099261,13cyclictest0-21swapper/200:20:252
3104099261,12cyclictest0-21swapper/219:25:412
3104099261,12cyclictest0-21swapper/200:10:532
3104099261,11cyclictest0-21swapper/221:00:182
3104099260,12cyclictest0-21swapper/222:01:252
3102799263,11cyclictest0-21swapper/019:25:010
3102799261,4cyclictest12723-21missed_timers22:05:180
3104099256,3cyclictest0-21swapper/223:25:102
3104099251,9cyclictest0-21swapper/200:07:012
3104099251,22cyclictest0-21swapper/222:50:522
3104099251,12cyclictest0-21swapper/223:34:562
3104099251,12cyclictest0-21swapper/221:25:522
3104099250,9cyclictest0-21swapper/220:06:392
3104099250,12cyclictest0-21swapper/221:46:282
3104099250,11cyclictest0-21swapper/220:54:142
3103699254,16cyclictest13548-40dnf22:07:261
3102799252,11cyclictest7813-21perf23:10:010
185332250,3sleep011-21rcuc/022:20:110
3104099241,14cyclictest0-21swapper/219:44:412
3104099241,12cyclictest0-21swapper/223:40:522
3104099241,12cyclictest0-21swapper/222:25:372
3104099241,12cyclictest0-21swapper/221:22:042
3104099241,10cyclictest0-21swapper/221:50:172
3104099240,11cyclictest0-21swapper/219:21:102
3104099240,10cyclictest0-21swapper/222:14:472
3103699243,7cyclictest24755-21latency_hist22:35:011
31036992412,6cyclictest26664-21kworker/u8:220:44:281
3104699231,16cyclictest0-21swapper/300:35:283
3104099235,3cyclictest0-21swapper/220:28:272
3104099231,7cyclictest0-21swapper/220:00:032
31040992315,3cyclictest0-21swapper/200:16:132
3104099231,20cyclictest0-21swapper/223:08:242
3104099231,14cyclictest0-21swapper/220:46:342
3104099231,14cyclictest0-21swapper/220:46:342
3103699237,7cyclictest21315-21munin-run20:00:011
3103699232,9cyclictest23-21ksoftirqd/100:20:011
3103699231,3cyclictest25728-21munin-run20:10:011
3103699231,19cyclictest5836-21chrt19:23:511
31027992317,3cyclictest18167-21munin-run22:20:010
3102799230,6cyclictest0-21swapper/023:39:590
3104699227,4cyclictest839-21lldpd22:00:163
3104699222,6cyclictest27575-21munin-run23:55:013
3104699222,11cyclictest6165-21/usr/sbin/munin19:25:153
3104099227,3cyclictest0-21swapper/220:17:092
3104099221,8cyclictest0-21swapper/220:32:032
3104099221,20cyclictest0-21swapper/223:40:002
3104099220,11cyclictest0-21swapper/222:19:572
3103699222,17cyclictest221ktimersoftd/122:45:181
31036992218,3cyclictest0-21swapper/121:36:501
3103699221,19cyclictest0-21swapper/123:26:361
3103699221,19cyclictest0-21swapper/122:10:201
3102799225,15cyclictest19135-21perf19:55:010
3102799221,9cyclictest21882-21kworker/0:221:18:170
3102799221,11cyclictest26348-21perf21:25:020
3252210,1sleep30-21swapper/320:25:133
3104699212,10cyclictest31234-21/usr/sbin/munin19:10:143
3104699211,11cyclictest6268-21ntp_states21:50:213
3104699211,11cyclictest23749-21/usr/sbin/munin20:05:253
3104099211,12cyclictest0-21swapper/222:59:242
3104099211,11cyclictest0-21swapper/221:08:582
3104099210,9cyclictest0-21swapper/219:45:532
3104099210,14cyclictest0-21swapper/200:04:572
3104099210,13cyclictest0-21swapper/222:44:292
3103699219,3cyclictest11112-21perl19:35:161
3103699216,12cyclictest0-21swapper/119:16:561
3103699212,17cyclictest0-21swapper/120:03:591
3103699212,11cyclictest16544-21diskstats22:15:161
3103699211,9cyclictest0-21swapper/121:33:291
31036992116,3cyclictest25527-21fschecks_time22:35:171
3103699211,18cyclictest0-21swapper/100:11:021
3102799217,5cyclictest0-21swapper/022:51:370
3102799217,5cyclictest0-21swapper/021:43:410
3102799216,6cyclictest0-21swapper/021:12:210
3102799214,3cyclictest87350irq/126-enp1s0-22:00:510
3102799211,4cyclictest0-21swapper/000:24:020
3102799211,4cyclictest0-21swapper/000:24:020
3102799211,17cyclictest4974-21munin-run21:50:010
268802210,4sleep03102799cyclictest20:10:170
3104699208,3cyclictest8976-21/usr/sbin/munin20:45:283
3104699208,3cyclictest8976-21/usr/sbin/munin20:45:283
3104699206,4cyclictest0-21swapper/322:35:023
3104699205,5cyclictest18670-21unixbench_multi19:50:263
3104699203,14cyclictest12519-21idleruntime-cro19:40:003
3104699201,6cyclictest0-21swapper/321:34:343
31046992014,3cyclictest14651-21gltestperf22:10:163
3104699201,2cyclictest28788-21grep23:55:183
3104699201,10cyclictest16809-21/usr/sbin/munin23:30:123
3104699201,10cyclictest0-21swapper/321:25:023
3104699201,10cyclictest0-21swapper/320:32:593
3104699200,9cyclictest0-21swapper/323:26:583
3104099208,3cyclictest0-21swapper/223:50:022
3104099207,3cyclictest0-21swapper/219:17:542
3104099206,3cyclictest0-21swapper/223:22:122
3104099204,3cyclictest0-21swapper/220:37:552
3104099201,12cyclictest0-21swapper/223:15:482
3104099201,11cyclictest0-21swapper/220:22:392
3103699205,3cyclictest16035-21sort21:00:151
3103699205,3cyclictest0-21swapper/122:22:361
3103699203,8cyclictest87550irq/128-enp1s0-23:10:231
3103699203,8cyclictest87550irq/128-enp1s0-23:10:231
3103699202,3cyclictest0-21swapper/123:46:361
3103699202,2cyclictest4678-21kworker/u8:121:51:011
31036992016,3cyclictest0-21swapper/123:24:421
31036992016,3cyclictest0-21swapper/121:23:351
31036992015,3cyclictest22836-21sed21:15:171
31036992015,3cyclictest2021-21latency_hist00:10:021
3103699201,3cyclictest0-21swapper/119:47:221
3103699201,18cyclictest0-21swapper/120:56:131
3103699201,18cyclictest0-21swapper/120:14:261
3103699201,17cyclictest0-21swapper/119:40:251
3103699201,17cyclictest0-21swapper/119:10:191
3103699201,16cyclictest31572-21runrttasks22:50:041
3103699201,11cyclictest827-21runrttasks20:40:001
3103699201,11cyclictest3731-21cut23:00:131
3103699201,11cyclictest28147-21diskstats23:55:141
3103699201,11cyclictest17210-21cpu19:50:111
3103699201,10cyclictest8637-21cat00:25:021
3103699201,10cyclictest8637-21cat00:25:011
3103699201,10cyclictest25745-21cpuspeed_turbos23:50:141
3103699201,10cyclictest0-21swapper/122:44:271
3103699200,3cyclictest0-21swapper/121:25:151
3102799209,3cyclictest25605-21/usr/sbin/munin23:50:170
3102799208,3cyclictest6261-21ntp_states21:50:200
3102799208,3cyclictest24476-21ntp_states23:45:240
3102799207,4cyclictest0-21swapper/022:55:250
3102799205,6cyclictest755-21rngd22:40:220
3102799205,2cyclictest87350irq/126-enp1s0-22:48:490
3102799203,2cyclictest87350irq/126-enp1s0-23:00:190
3102799201,11cyclictest19740-21diskmemload00:09:320
3102799201,10cyclictest11495-21awk23:15:250
224672201,2sleep30-21swapper/322:28:593
59242191,2sleep30-21swapper/300:16:053
35812191,2sleep10-21swapper/120:30:251
3104699195,4cyclictest12193-21fschecks_count22:05:143
3104699191,9cyclictest0-21swapper/319:55:133
3104699191,8cyclictest0-21swapper/321:45:253
3104699191,7cyclictest0-21swapper/321:38:003
3104699191,4cyclictest0-21swapper/322:18:543
31046991913,3cyclictest16535-21users19:45:263
31046991912,4cyclictest27824-21sed22:40:153
3104699191,12cyclictest0-21swapper/320:35:153
3104699191,10cyclictest0-21swapper/323:23:503
3104699191,10cyclictest0-21swapper/300:04:373
3104099197,3cyclictest0-21swapper/221:57:392
3104099197,3cyclictest0-21swapper/221:57:392
3104099197,3cyclictest0-21swapper/219:12:272
3104099193,3cyclictest0-21swapper/220:55:562
3103699197,4cyclictest0-21swapper/119:29:201
3103699197,4cyclictest0-21swapper/100:35:131
3103699192,3cyclictest0-21swapper/123:35:571
3103699192,3cyclictest0-21swapper/100:28:031
3103699192,10cyclictest21-21rcuc/120:50:241
31036991916,2cyclictest21-21rcuc/123:40:171
31036991916,2cyclictest0-21swapper/122:25:381
31036991915,3cyclictest0-21swapper/123:09:561
31036991914,3cyclictest555-21grep20:25:151
3103699191,3cyclictest0-21swapper/120:19:231
3103699191,16cyclictest0-21swapper/123:18:341
3103699191,16cyclictest0-21swapper/120:20:471
3103699191,12cyclictest0-21swapper/100:33:501
3103699191,10cyclictest8657-21smart_sda21:55:241
3103699191,10cyclictest8657-21smart_sda21:55:241
3103699190,18cyclictest0-21swapper/121:49:491
3102799198,3cyclictest30328-21df00:00:140
3102799192,9cyclictest755-21rngd23:26:590
3102799191,8cyclictest0-21swapper/023:30:240
3102799191,6cyclictest0-21swapper/000:17:560
3102799191,11cyclictest0-21swapper/021:55:210
3102799191,11cyclictest0-21swapper/021:55:210
3102799191,10cyclictest22789-21/usr/sbin/munin22:30:110
3102799191,10cyclictest13057-21/usr/sbin/munin00:35:230
3102799191,10cyclictest0-21swapper/022:10:280
3102799190,11cyclictest0-21swapper/023:45:020
125352190,3sleep00-21swapper/000:31:230
121632191,3sleep30-21swapper/323:19:513
3104699187,2cyclictest7394-21/usr/sbin/munin21:55:133
3104699187,2cyclictest7394-21/usr/sbin/munin21:55:133
3104699185,4cyclictest19740-21diskmemload22:50:213
3104699181,9cyclictest808-21snmpd23:35:283
3104699181,8cyclictest0-21swapper/323:50:013
3104699181,6cyclictest0-21swapper/323:08:293
3104699181,6cyclictest0-21swapper/321:20:533
3104699181,4cyclictest20350-21sshd22:24:593
31046991812,3cyclictest2900-21grep22:55:253
3104699181,14cyclictest30899-21tail22:45:273
3104699180,9cyclictest0-21swapper/321:05:173
3103699181,9cyclictest31162-21pmu-power00:00:321
31036991814,3cyclictest0-21swapper/120:47:541
31036991814,3cyclictest0-21swapper/120:47:541
3103699181,2cyclictest0-21swapper/122:02:191
31036991812,3cyclictest20089-21cpuspeed_turbos21:10:121
3103699181,16cyclictest0-21swapper/123:33:011
3103699181,16cyclictest0-21swapper/122:58:191
3103699181,16cyclictest0-21swapper/119:34:491
3103699180,16cyclictest0-21swapper/121:40:421
3102799183,8cyclictest87550irq/128-enp1s0-19:10:160
3102799183,12cyclictest8361-21/usr/sbin/munin19:30:140
3102799182,7cyclictest30339-21/usr/sbin/munin20:20:230
3102799182,4cyclictest10558-21/usr/sbin/munin19:35:240
3102799181,5cyclictest27798-21/usr/sbin/munin23:55:180
3102799181,4cyclictest20337-21needreboot19:55:190
3102799181,3cyclictest21442-21grep22:25:180
31027991813,3cyclictest6554-21munin-run20:40:000
3102799181,15cyclictest0-21swapper/020:06:110
3102799181,11cyclictest0-21swapper/021:35:400
3102799181,10cyclictest9744-21sshd23:11:580
3102799181,10cyclictest9744-21sshd23:11:580
31027991810,6cyclictest26664-21kworker/u8:221:05:200
3102799180,6cyclictest0-21swapper/000:12:170
3104699171,9cyclictest808-21snmpd00:30:393
3104699171,7cyclictest0-21swapper/319:20:153
3104699171,6cyclictest8361-21/usr/sbin/munin19:30:203
3104699171,6cyclictest6777-21/usr/sbin/munin20:40:153
3104699171,6cyclictest10221-21unixbench_singl00:25:263
3104699171,6cyclictest0-21swapper/320:20:213
3104699171,6cyclictest0-21swapper/320:15:513
3104699171,6cyclictest0-21swapper/319:15:183
3104699171,5cyclictest0-21swapper/322:36:243
3104699171,2cyclictest0-21swapper/323:12:483
3104699171,2cyclictest0-21swapper/323:12:473
31046991712,3cyclictest26559-21tail20:10:143
31046991712,3cyclictest15900-21df21:00:143
31046991712,3cyclictest1368-21gltestperf21:40:163
31046991711,3cyclictest3635-21cpu23:00:133
31046991711,3cyclictest2756-21hwlatdetect00:10:163
3104699171,13cyclictest23312-21pmu-power21:15:233
31046991711,3cyclictest20232-21cat21:10:133
3104699171,13cyclictest1988-21wc00:10:013
3104699171,13cyclictest14100-21wc19:40:243
3104699170,14cyclictest0-21swapper/300:20:233
3104699170,14cyclictest0-21swapper/300:20:223
3102799176,3cyclictest12691-21expr23:20:140
3102799175,3cyclictest0-21swapper/000:25:020
3102799171,9cyclictest839-21lldpd19:42:150
3102799171,9cyclictest0-21swapper/021:28:460
3102799171,3cyclictest0-21swapper/020:42:530
3102799171,3cyclictest0-21swapper/020:00:250
3102799171,2cyclictest0-21swapper/021:00:020
31027991712,3cyclictest14132-21if_enp2s020:55:170
3102799171,14cyclictest0-21swapper/020:25:070
3102799171,10cyclictest0-21swapper/019:45:200
229222171,2sleep30-21swapper/323:41:543
3104699161,6cyclictest21539-21/usr/sbin/munin20:00:243
31046991610,4cyclictest14063-21idleruntime20:55:173
3102799169,1cyclictest0-21swapper/020:15:230
3102799161,3cyclictest0-21swapper/019:15:460
31027991612,3cyclictest0-21swapper/020:30:350
3102799159,3cyclictest9088-21cpuspeed_turbos20:45:140
3102799159,3cyclictest9088-21cpuspeed_turbos20:45:140
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional