You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-13 - 18:05

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Sat Dec 13, 2025 12:44:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
40382239165,23sleep20-21swapper/207:09:582
20732206169,24sleep30-21swapper/307:05:043
37902201165,25sleep00-21swapper/007:06:460
37662200165,24sleep10-21swapper/107:06:271
4130999087,2cyclictest0-21swapper/110:30:401
172272500,2sleep30-21swapper/312:30:183
413499411,14cyclictest0-21swapper/209:32:302
155552410,1sleep00-21swapper/012:25:260
413499391,13cyclictest0-21swapper/208:30:062
413099391,27cyclictest0-21swapper/107:45:161
414099371,17cyclictest1208-21missed_timers10:40:223
413499361,12cyclictest0-21swapper/212:37:222
413499361,12cyclictest0-21swapper/212:37:222
413099361,13cyclictest0-21swapper/109:48:201
4140993517,7cyclictest0-21swapper/309:20:373
414099351,14cyclictest2017-21/usr/sbin/munin08:15:183
413499331,23cyclictest0-21swapper/211:41:342
413499331,21cyclictest0-21swapper/208:12:462
414099321,24cyclictest0-21swapper/309:27:493
414099321,21cyclictest0-21swapper/311:46:303
413499321,23cyclictest0-21swapper/211:17:342
412299324,13cyclictest8842-21awk11:00:020
414099317,9cyclictest0-21swapper/309:19:223
414099311,21cyclictest0-21swapper/308:13:103
413499311,21cyclictest0-21swapper/211:23:382
413499311,20cyclictest0-21swapper/208:00:382
413499311,20cyclictest0-21swapper/208:00:382
413499311,13cyclictest0-21swapper/207:26:532
413499311,10cyclictest29782-21latency_hist09:20:022
413099311,8cyclictest0-21swapper/108:54:041
413099310,13cyclictest0-21swapper/108:31:151
4140993018,3cyclictest0-21swapper/307:59:583
414099301,15cyclictest0-21swapper/311:40:563
4140993010,8cyclictest0-21swapper/311:17:453
4140993010,7cyclictest0-21swapper/312:27:463
413499301,12cyclictest0-21swapper/211:49:582
413499301,12cyclictest0-21swapper/207:38:492
413499301,11cyclictest0-21swapper/211:06:582
413099301,2cyclictest0-21swapper/112:17:281
413099301,12cyclictest0-21swapper/108:45:081
413099301,10cyclictest0-21swapper/108:18:481
413099300,12cyclictest0-21swapper/109:36:161
413099300,11cyclictest0-21swapper/112:30:441
413099300,11cyclictest0-21swapper/112:29:521
59382290,3sleep29-21rcu_preempt08:25:132
414099291,19cyclictest0-21swapper/308:27:243
414099291,16cyclictest18163-21missed_timers10:05:203
414099291,12cyclictest0-21swapper/312:20:083
414099291,12cyclictest0-21swapper/312:14:123
414099291,12cyclictest0-21swapper/310:37:363
413499291,13cyclictest0-21swapper/209:09:492
413499291,13cyclictest0-21swapper/208:42:302
413499291,12cyclictest0-21swapper/210:19:342
413499291,12cyclictest0-21swapper/207:32:412
413499291,10cyclictest0-21swapper/211:52:222
413099291,9cyclictest0-21swapper/111:44:481
413099291,13cyclictest0-21swapper/112:07:001
413099291,12cyclictest0-21swapper/110:43:361
4122992910,16cyclictest18586-21munin-run08:55:010
414099288,9cyclictest0-21swapper/310:50:223
414099281,14cyclictest0-21swapper/309:05:363
414099281,12cyclictest26093-21hddtemp_smartct09:10:163
414099280,11cyclictest0-21swapper/307:33:263
413499283,9cyclictest0-21swapper/208:15:182
413499281,25cyclictest0-21swapper/207:59:582
413499281,19cyclictest0-21swapper/212:15:062
413499281,13cyclictest0-21swapper/208:52:142
413499281,13cyclictest0-21swapper/208:37:502
413499281,12cyclictest0-21swapper/207:50:062
413499281,12cyclictest0-21swapper/207:19:342
413099281,9cyclictest0-21swapper/111:54:521
413099281,21cyclictest0-21swapper/110:01:181
413099281,17cyclictest0-21swapper/109:02:041
413099281,16cyclictest0-21swapper/112:36:591
413099281,16cyclictest0-21swapper/112:36:591
413099281,11cyclictest0-21swapper/111:48:081
413099281,11cyclictest0-21swapper/111:31:121
413099281,11cyclictest0-21swapper/111:18:081
413099281,11cyclictest0-21swapper/109:34:041
413099281,11cyclictest0-21swapper/107:28:081
413099281,10cyclictest0-21swapper/110:26:271
413099281,10cyclictest0-21swapper/109:53:281
413099280,12cyclictest0-21swapper/110:21:001
413099280,12cyclictest0-21swapper/110:20:591
413099280,11cyclictest0-21swapper/111:06:321
413099280,11cyclictest0-21swapper/109:13:521
413099280,10cyclictest0-21swapper/112:10:321
412299282,14cyclictest6019-21munin-run09:40:010
186822280,3sleep123-21ksoftirqd/111:20:181
414099279,9cyclictest0-21swapper/311:14:303
414099279,7cyclictest0-21swapper/308:59:263
414099278,8cyclictest0-21swapper/308:06:463
414099278,8cyclictest0-21swapper/308:04:303
414099278,8cyclictest0-21swapper/308:04:303
414099271,13cyclictest0-21swapper/312:35:263
414099271,13cyclictest0-21swapper/312:35:263
414099271,12cyclictest0-21swapper/310:45:243
414099271,12cyclictest0-21swapper/309:58:443
414099271,12cyclictest0-21swapper/308:38:363
4140992710,3cyclictest13585-21/usr/sbin/munin07:30:023
413499271,13cyclictest6295-21latency_hist07:15:022
413499271,13cyclictest0-21swapper/212:12:142
413499271,13cyclictest0-21swapper/209:24:142
413499271,12cyclictest0-21swapper/210:40:342
413499271,12cyclictest0-21swapper/209:52:382
413499271,12cyclictest0-21swapper/209:02:452
413499271,12cyclictest0-21swapper/207:40:302
413499271,11cyclictest0-21swapper/207:24:422
413499271,10cyclictest0-21swapper/210:20:382
413499271,10cyclictest0-21swapper/210:20:382
413099271,9cyclictest0-21swapper/111:12:121
413099271,8cyclictest0-21swapper/110:09:361
413099271,13cyclictest0-21swapper/107:41:281
413099271,12cyclictest0-21swapper/110:53:281
413099271,11cyclictest0-21swapper/109:59:561
413099271,11cyclictest0-21swapper/109:16:301
413099270,11cyclictest0-21swapper/110:12:081
413099270,10cyclictest0-21swapper/112:22:481
413099270,10cyclictest0-21swapper/109:29:201
412299271,10cyclictest20539-21kworker/0:009:07:260
412299271,10cyclictest14073-21kworker/0:308:55:020
4122992710,14cyclictest28973-21munin-run08:05:010
4122992710,14cyclictest28973-21munin-run08:05:000
4140992613,4cyclictest6437-21munin-node08:31:103
414099261,10cyclictest831-21runrttasks09:51:323
413099261,11cyclictest0-21swapper/112:02:361
413099261,11cyclictest0-21swapper/111:55:481
413099261,10cyclictest0-21swapper/107:12:561
413099260,9cyclictest0-21swapper/109:21:361
412299262,10cyclictest27610-21kworker/0:009:24:180
412299261,10cyclictest10039-21kworker/0:207:23:460
414099258,6cyclictest0-21swapper/310:00:423
414099255,7cyclictest0-21swapper/307:23:383
414099251,17cyclictest0-21swapper/310:21:103
414099251,17cyclictest0-21swapper/310:21:103
414099251,16cyclictest0-21swapper/311:23:433
414099251,15cyclictest0-21swapper/307:50:243
414099251,12cyclictest0-21swapper/310:58:303
414099251,10cyclictest0-21swapper/309:00:123
4140992510,7cyclictest0-21swapper/307:18:263
414099250,12cyclictest0-21swapper/307:42:303
413499251,15cyclictest0-21swapper/209:55:262
413499251,14cyclictest0-21swapper/212:30:102
413499251,12cyclictest0-21swapper/210:30:502
413499251,12cyclictest0-21swapper/210:08:022
413499251,12cyclictest0-21swapper/208:59:332
413499251,11cyclictest0-21swapper/212:26:342
413499251,10cyclictest0-21swapper/208:21:222
413499251,10cyclictest0-21swapper/208:09:342
413099251,9cyclictest0-21swapper/108:24:281
413099251,9cyclictest0-21swapper/107:52:161
413099251,8cyclictest0-21swapper/107:24:311
413099251,11cyclictest0-21swapper/111:37:401
413099251,11cyclictest0-21swapper/108:00:481
413099251,11cyclictest0-21swapper/108:00:481
413099250,9cyclictest0-21swapper/109:09:241
413099250,8cyclictest0-21swapper/107:15:521
413099250,10cyclictest0-21swapper/111:03:121
413099250,10cyclictest0-21swapper/110:46:001
413099250,10cyclictest0-21swapper/108:42:391
412299252,12cyclictest1618-21munin-run09:30:010
414099247,3cyclictest0-21swapper/311:50:263
414099245,3cyclictest0-21swapper/307:45:573
4140992413,3cyclictest0-21swapper/308:21:423
414099241,12cyclictest0-21swapper/312:15:503
414099241,12cyclictest0-21swapper/311:39:243
414099241,11cyclictest0-21swapper/309:30:353
414099240,11cyclictest0-21swapper/310:33:443
413499241,16cyclictest0-21swapper/209:48:422
413499241,12cyclictest0-21swapper/210:12:422
413499241,10cyclictest0-21swapper/210:59:052
413099241,11cyclictest0-21swapper/110:38:391
413099240,12cyclictest0-21swapper/108:25:191
412299241,16cyclictest3290-21kworker/0:107:27:220
412299241,12cyclictest15447-21perf11:15:010
412299241,11cyclictest16805-21kworker/0:210:10:010
412299241,10cyclictest27610-21kworker/0:009:46:420
4122992410,3cyclictest11017-21kworker/0:111:26:500
414099239,3cyclictest0-21swapper/311:27:373
414099237,3cyclictest0-21swapper/308:42:223
414099237,3cyclictest0-21swapper/307:12:183
414099236,3cyclictest0-21swapper/310:11:463
414099231,16cyclictest0-21swapper/310:15:503
414099231,14cyclictest0-21swapper/308:47:203
4140992310,3cyclictest0-21swapper/309:37:333
414099230,13cyclictest0-21swapper/312:00:043
413499239,3cyclictest0-21swapper/212:21:582
413499231,14cyclictest0-21swapper/207:49:302
413499231,12cyclictest0-21swapper/210:29:502
413499231,11cyclictest0-21swapper/209:41:582
413499231,10cyclictest0-21swapper/211:27:102
413499231,10cyclictest0-21swapper/211:10:102
413499231,10cyclictest0-21swapper/210:51:012
413499231,10cyclictest0-21swapper/209:13:582
413099231,9cyclictest0-21swapper/109:44:521
413099231,8cyclictest0-21swapper/108:14:161
413099231,16cyclictest0-21swapper/110:15:251
413099230,16cyclictest0-21swapper/108:36:431
413099230,10cyclictest0-21swapper/108:05:241
412299239,2cyclictest27610-21kworker/0:010:03:330
412299237,13cyclictest22993-21munin-run09:05:010
412299232,13cyclictest858-21kworker/0:008:15:220
412299232,11cyclictest10025-21kworker/0:108:40:100
412299231,14cyclictest3290-21kworker/0:107:30:220
412299231,12cyclictest19986-21kworker/0:207:56:460
412299231,12cyclictest17731-21kworker/0:007:43:420
412299231,12cyclictest16805-21kworker/0:210:05:140
6112220,5sleep0412299cyclictest10:40:160
4140992214,2cyclictest0-21swapper/312:05:223
414099221,13cyclictest0-21swapper/311:08:263
414099221,13cyclictest0-21swapper/308:52:403
414099221,12cyclictest0-21swapper/309:42:303
414099220,13cyclictest0-21swapper/309:49:103
413499221,15cyclictest0-21swapper/211:34:502
413499221,13cyclictest0-21swapper/209:29:462
413499221,11cyclictest5289-21/usr/sbin/munin12:05:162
413499221,10cyclictest0-21swapper/210:45:492
413099226,3cyclictest0-21swapper/107:58:461
413099221,9cyclictest0-21swapper/111:29:241
413099221,10cyclictest0-21swapper/110:55:441
413099220,8cyclictest0-21swapper/108:56:161
413099220,8cyclictest0-21swapper/107:31:161
412299229,2cyclictest27610-21kworker/0:009:59:130
412299227,8cyclictest30982-21kworker/0:208:27:580
412299222,8cyclictest15839-21kworker/0:212:30:380
412299222,11cyclictest21887-21kworker/0:211:34:100
412299222,10cyclictest30982-21kworker/0:208:31:020
412299222,10cyclictest22877-21kworker/0:109:10:060
412299222,10cyclictest17421-21kworker/0:211:17:580
412299221,12cyclictest3290-21kworker/0:107:12:220
412299221,11cyclictest30982-21kworker/0:208:08:370
414099219,3cyclictest0-21swapper/311:59:583
414099216,7cyclictest770-21rngd11:32:463
414099211,3cyclictest0-21swapper/307:35:583
413499214,8cyclictest0-21swapper/211:01:332
413499211,13cyclictest0-21swapper/211:35:542
413499211,10cyclictest0-21swapper/210:39:182
413499210,11cyclictest0-21swapper/210:00:262
412299217,9cyclictest11017-21kworker/0:111:06:180
412299217,8cyclictest7227-21kworker/0:112:17:300
412299217,8cyclictest3290-21kworker/0:107:38:140
412299217,8cyclictest19986-21kworker/0:207:49:540
412299217,4cyclictest7227-21kworker/0:112:23:140
412299217,2cyclictest2571-21kworker/0:012:04:470
412299216,4cyclictest30036-21kworker/0:210:45:140
412299216,3cyclictest7227-21kworker/0:112:12:580
412299212,8cyclictest10025-21kworker/0:108:38:180
412299212,7cyclictest24162-21kworker/0:011:40:340
412299212,11cyclictest28837-21/usr/sbin/munin11:45:200
412299212,10cyclictest24192-21kworker/0:311:36:020
412299211,10cyclictest26657-21tr10:25:170
412299211,10cyclictest22877-21kworker/0:109:16:260
412299211,10cyclictest1482-21kworker/0:109:32:100
414099202,5cyclictest26028-21/usr/sbin/munin10:25:243
412299207,8cyclictest3290-21kworker/0:107:16:570
412299207,2cyclictest12573-21kworker/0:008:46:180
412299206,8cyclictest2571-21kworker/0:012:08:330
412299202,8cyclictest11017-21kworker/0:111:22:060
412299202,10cyclictest27610-21kworker/0:009:51:410
412299201,12cyclictest2215-21kworker/0:010:53:180
412299201,11cyclictest32668-21kworker/0:111:56:210
412299201,10cyclictest3108-21kworker/0:108:21:100
412299201,10cyclictest31032-21/usr/sbin/munin11:50:240
412299201,10cyclictest30982-21kworker/0:208:10:090
412299201,10cyclictest19986-21kworker/0:207:52:300
412299200,10cyclictest30036-21kworker/0:210:39:380
414099191,14cyclictest0-21swapper/311:03:123
413499191,3cyclictest1790-21memory11:55:192
413499191,11cyclictest0-21swapper/208:46:102
413499191,10cyclictest0-21swapper/209:38:462
413099190,14cyclictest0-21swapper/107:39:291
412299198,3cyclictest19071-21kworker/0:110:30:220
412299191,10cyclictest21301-21kworker/0:010:16:220
412299190,16cyclictest27610-21kworker/0:009:40:140
413499180,4cyclictest0-21swapper/212:00:262
412299187,4cyclictest15839-21kworker/0:212:38:020
412299187,4cyclictest15839-21kworker/0:212:38:020
412299183,8cyclictest8479-21kworker/0:211:03:300
412299181,9cyclictest800-21snmpd10:20:160
412299181,9cyclictest800-21snmpd10:20:160
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional