You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-15 - 03:35

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack5slot4s.osadl.org (updated Mon Jun 15, 2026 00:44:14)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
225151721990,2sleep3471rcuc/319:55:363
237313321970,5sleep3471rcuc/322:00:273
2202146998575,7cyclictest2170309-21kworker/u8:3+flush-8:019:25:441
2202141998479,3cyclictest2288071-21kworker/u8:2+events_unbound21:00:010
2202146998073,5cyclictest2170309-21kworker/u8:3+events_unbound20:00:431
23138872790,4sleep3481ktimers/321:00:333
2202146997465,6cyclictest1353594-21kworker/u8:0+flush-8:020:25:421
2202146997363,7cyclictest2262572-21ntpq20:05:391
2202146997163,6cyclictest2502146-21kworker/u8:1+flush-8:000:10:461
2202141997164,6cyclictest2390865-21kworker/u8:3+events_unbound23:40:440
2202146997064,4cyclictest2201995-21kworker/u8:1+events_unbound20:55:451
2202146997064,4cyclictest2170309-21kworker/u8:3+flush-8:020:10:471
2202146997063,5cyclictest2332476-21kworker/u8:1+events_unbound21:40:451
2202146996961,6cyclictest2288071-21kworker/u8:2+events_unbound21:50:461
2202146996861,5cyclictest2288071-21kworker/u8:2+flush-8:022:00:451
2202146996861,5cyclictest1353594-21kworker/u8:0+events_unbound21:00:341
2202141996861,5cyclictest2170309-21kworker/u8:3+events_unbound19:30:260
2202141996655,8cyclictest2531207-21ntpq00:35:370
2202146996459,3cyclictest2367353-21kworker/u8:0+events_unbound22:15:451
2202146996358,4cyclictest2288071-21kworker/u8:2+flush-8:020:45:431
2202146996253,7cyclictest2223963-21/usr/sbin/munin19:30:351
2202141996251,9cyclictest2426568-21ntpq22:50:380
2202146996150,8cyclictest2481450-21ntpq23:45:381
2202141996149,9cyclictest2348143-21diskstats21:35:280
24571622600,1chrt471rcuc/323:20:463
2202146996054,4cyclictest2492746-21kworker/u8:4+events_unbound00:05:431
2202146996053,5cyclictest2412334-21kworker/u8:2+flush-8:022:55:431
2202146996051,7cyclictest2403018-21/usr/sbin/munin22:30:431
2202146996051,7cyclictest2403018-21/usr/sbin/munin22:30:431
2202146995952,5cyclictest2170309-21kworker/u8:3+events_unbound19:20:281
2202146995951,5cyclictest2201995-21kworker/u8:1+events_unbound19:35:291
2202141995951,6cyclictest1353594-21kworker/u8:0+events_unbound21:00:430
2202146995852,4cyclictest2412334-21kworker/u8:2+flush-8:000:00:311
2202146995751,4cyclictest2170309-21kworker/u8:3+flush-8:019:10:421
2202146995750,5cyclictest2502146-21kworker/u8:1+events_unbound00:35:421
2202146995749,6cyclictest2201995-21kworker/u8:1+flush-8:019:15:451
2202146995748,6cyclictest1353594-21kworker/u8:0+events_unbound20:15:381
2202141995748,7cyclictest2253845-21/usr/sbin/munin20:00:370
2202146995650,4cyclictest2407216-21kworker/u8:0+events_unbound22:45:281
2202146995650,4cyclictest2288071-21kworker/u8:2+flush-8:021:05:301
2202146995650,4cyclictest2170309-21kworker/u8:3+flush-8:019:50:311
2202146995648,6cyclictest2516493-21ntpq00:20:411
2202141995646,8cyclictest2501573-21ntpq00:05:410
2202146995551,3cyclictest2526345-21kworker/u8:3+events_unbound00:30:451
2202146995551,3cyclictest2332476-21kworker/u8:1+flush-8:022:05:291
2202146995551,3cyclictest2288071-21kworker/u8:2+flush-8:022:10:311
2202146995448,4cyclictest2502146-21kworker/u8:1+flush-8:000:15:301
2202146995447,6cyclictest2481521-21kworker/u8:0+flush-8:023:55:391
2202146995447,5cyclictest2390865-21kworker/u8:3+events_unbound22:50:421
2202146995446,6cyclictest2441708-21ntpq23:05:371
2202146995446,6cyclictest2288071-21kworker/u8:2+flush-8:021:25:421
2202141995446,6cyclictest2407961-21apt-get22:35:270
2202141995446,6cyclictest2407961-21apt-get22:35:260
2202141995445,7cyclictest2496613-21ntpq00:00:390
2202141995342,8cyclictest2247750-21ntpq19:50:400
2202147995249,2cyclictest2332476-21kworker/u8:1+events_unbound21:40:282
2202146995246,4cyclictest2170309-21kworker/u8:3+events_unbound20:30:281
2202146995242,8cyclictest2307141-21ntpq20:50:361
2202146995146,3cyclictest2407216-21kworker/u8:0+events_unbound22:40:271
2202146995143,6cyclictest2482631-21/usr/sbin/munin23:50:381
2202146995142,7cyclictest2352635-21gpgv21:40:001
2202141995142,6cyclictest2327836-21latency_hist21:15:010
2202146995045,4cyclictest1353594-21kworker/u8:0+events_unbound19:40:441
2202146995045,4cyclictest1353594-21kworker/u8:0+events_unbound19:40:441
2202141995042,6cyclictest2292343-21ntpq20:35:400
2202141995040,8cyclictest2366763-21ntpq21:50:390
2202141995039,8cyclictest2526259-21ntpq00:30:390
2202146994944,4cyclictest2288071-21kworker/u8:2+flush-8:020:40:461
2202146994944,3cyclictest2412334-21kworker/u8:2+flush-8:000:25:371
2202146994943,4cyclictest2397224-21kworker/u8:1+events_unbound22:25:001
2202146994940,6cyclictest2412334-21kworker/u8:2+events_unbound23:00:401
2202146994939,7cyclictest2312678-21kworker/u8:3+flush-8:021:10:411
2202146994845,2cyclictest2390865-21kworker/u8:3+flush-8:023:40:441
2202146994844,3cyclictest2288071-21kworker/u8:2+events_unbound22:25:401
2202146994842,5cyclictest2412334-21kworker/u8:2+flush-8:023:30:421
2202146994842,4cyclictest1353594-21kworker/u8:0+events_unbound19:55:331
2202146994840,6cyclictest2412334-21kworker/u8:2+events_unbound23:35:431
2202146994840,6cyclictest2332476-21kworker/u8:1+events_unbound21:30:341
2202141994838,7cyclictest2401890-21ntpq22:25:390
2202146994639,5cyclictest2390865-21kworker/u8:3+flush-8:022:35:421
2202146994639,5cyclictest2390865-21kworker/u8:3+flush-8:022:35:411
2202146994636,8cyclictest2371775-21ntpq21:55:411
2202141994641,3cyclictest2436735-21ntpq23:00:380
2202147994539,4cyclictest2447683-21cp23:15:002
2202147994538,5cyclictest2353041-21cp21:40:012
2202147994536,7cyclictest2492251-21apt-get00:00:012
2202146994437,5cyclictest2412334-21kworker/u8:2+flush-8:023:20:451
2202146994436,6cyclictest2331930-21ntpq21:15:401
2202146994335,6cyclictest2412334-21kworker/u8:2+flush-8:023:25:391
2202146994237,4cyclictest2412334-21kworker/u8:2+flush-8:023:15:341
2202146994235,5cyclictest2333091-21apt-get21:20:251
2202146994234,6cyclictest2170309-21kworker/u8:3+flush-8:020:20:351
2202141994233,7cyclictest2461760-21ntpq23:25:400
2202141994133,6cyclictest2447586-21latency_hist23:15:010
2202146994035,4cyclictest2412334-21kworker/u8:2+events_unbound23:10:431
2202141994034,4cyclictest2170309-21kworker/u8:3+flush-8:019:10:310
2202141993932,5cyclictest2233822-21latency_hist19:40:000
2202146993732,3cyclictest2201995-21kworker/u8:1+events_unbound20:35:261
2202141993732,3cyclictest2332789-21idleruntime-cro21:20:000
2202147993612,14cyclictest2378650-21sed22:05:322
2202146993631,3cyclictest2170309-21kworker/u8:3+flush-8:019:45:331
2202146993627,7cyclictest2358086-21apt-get21:45:261
2202141993632,2cyclictest2288071-21kworker/u8:2+flush-8:022:30:300
2202141993632,2cyclictest2288071-21kworker/u8:2+flush-8:022:30:300
2202141993631,4cyclictest2332476-21kworker/u8:1+flush-8:022:05:310
2202141993630,4cyclictest2412956-21apt-get22:40:240
2202141993628,6cyclictest2242807-21ntpq19:45:370
2202141993627,6cyclictest2517322-21latency_hist00:25:010
2202141993625,9cyclictest2471576-21ntpq23:35:350
23590522340,2sleep2371rcuc/221:45:342
2202141993424,7cyclictest2213515-21sshd19:15:460
2202147993224,6cyclictest2427517-21awk22:55:012
2202141993225,5cyclictest2511390-21needreboot00:15:380
2202141993221,8cyclictest1704890-21systemd-journal22:25:000
220214799316,12cyclictest39-21ksoftirqd/222:40:002
220214799316,12cyclictest39-21ksoftirqd/222:39:592
2202147993124,5cyclictest2228652-21apt-get19:35:002
2202141993125,4cyclictest2357806-21latency_hist21:45:010
2202141993120,8cyclictest1704890-21systemd-journal20:45:000
2202141993120,2cyclictest161rcu_preempt20:23:530
2202147993016,5cyclictest161rcu_preempt21:20:012
220214799300,28cyclictest1704890-21systemd-journal23:10:012
220214199309,12cyclictest2487848-21/usr/sbin/munin23:55:270
2202141993023,4cyclictest2347632-21true21:35:000
2202141993022,6cyclictest2201995-21kworker/u8:1+flush-8:020:05:440
2202141993017,10cyclictest1704890-21systemd-journal00:30:000
220214799297,6cyclictest2316121-21tr21:00:352
2202141992924,3cyclictest2368481-21irqcore21:55:330
2202141992924,3cyclictest2273030-21sshd20:15:460
2202141992919,8cyclictest2391693-21ntpq22:15:390
2202141992912,8cyclictest161rcu_preempt22:00:450
220214799286,6cyclictest2522518-21find00:30:252
2202147992824,2cyclictest2338338-21fschecks_time21:25:272
2202147992822,4cyclictest2387665-21gpgv22:15:012
2202147992813,2cyclictest161rcu_preempt20:05:332
220214799280,9cyclictest2466135-21cut23:30:362
220214799280,10cyclictest2497588-21grep00:05:002
220214199289,13cyclictest161rcu_preempt23:15:400
220214199288,5cyclictest2334389-21cat21:20:340
2202141992824,2cyclictest2422595-21systemd-run22:50:000
2202141992811,9cyclictest161rcu_preempt23:30:280
220214199280,26cyclictest2307610-21grep20:50:440
2202147992724,2cyclictest2219154-21aten2_r5power_p19:25:272
2202147992712,4cyclictest39-21ksoftirqd/200:35:272
220214799270,8cyclictest2347273-21seq21:30:472
2202141992722,3cyclictest2387402-21idleruntime-cro22:14:590
2202141992721,4cyclictest2438867-21kthreadcore23:05:320
2202141992718,7cyclictest1704890-21systemd-journal23:50:010
2202141992710,16cyclictest2337769-21kworker/0:0+events21:25:410
220214199270,26cyclictest2197111-21kworker/0:0+events19:25:390
220214799265,5cyclictest2390439-21tr22:15:342
220214799265,11cyclictest2215504-21cut19:20:332
2202147992622,3cyclictest2398415-21fschecks_time22:25:302
2202147992621,3cyclictest2473123-21fschecks_time23:40:292
220214799262,12cyclictest39-21ksoftirqd/220:00:372
220214799261,5cyclictest2372619-21munin-run22:00:002
220214799260,9cyclictest2518499-21grep00:25:362
220214799260,9cyclictest2512513-21latency_hist00:20:002
220214799260,9cyclictest2265280-21tr20:10:372
220214799260,7cyclictest2367675-21munin-plugin-st21:55:002
220214799260,16cyclictest2373509-21sed22:00:312
220214799260,10cyclictest2323234-21cut21:10:282
220214799260,10cyclictest2268798-21wget20:15:272
220214199266,5cyclictest2251033-21grep19:55:350
220214199263,8cyclictest589-21runrttasks20:10:280
220214199262,8cyclictest589-21runrttasks21:45:270
2202141992619,4cyclictest0-21swapper/021:10:010
2202141992616,8cyclictest1704890-21systemd-journal23:00:010
2202141992615,2cyclictest161rcu_preempt20:30:140
220214199260,15cyclictest2238815-21systemctl19:45:000
220214199260,15cyclictest2238815-21systemctl19:45:000
220214799258,9cyclictest0-21swapper/219:40:352
220214799258,9cyclictest0-21swapper/219:40:342
220214799256,6cyclictest39-21ksoftirqd/222:25:002
2202147992521,2cyclictest2458302-21fschecks_time23:25:292
2202147992519,4cyclictest2298700-21cpuspeed_turbos20:45:282
2202147992512,5cyclictest161rcu_preempt21:05:372
220214799250,9cyclictest2517473-21latency_hist00:25:012
220214799250,9cyclictest2448971-21sed23:15:322
220214799250,9cyclictest2403245-21sort22:30:282
220214799250,9cyclictest2403245-21sort22:30:272
220214799250,9cyclictest2335145-21tr21:20:352
220214799250,8cyclictest2305843-21cut20:50:322
220214799250,5cyclictest2229747-21ps19:35:342
220214799250,5cyclictest1-21systemd20:45:012
220214799250,14cyclictest2433742-21sed23:00:332
220214799250,14cyclictest2428636-21sed22:55:312
2202146992517,6cyclictest2201971-21latency_hist19:10:011
2202141992518,2cyclictest161rcu_preempt20:45:410
2202141992511,3cyclictest0-21swapper/000:10:340
220214799249,6cyclictest161rcu_preempt20:27:142
220214799242,6cyclictest39-21ksoftirqd/219:45:332
2202147992420,2cyclictest2249050-21aten2_r5power_v19:55:282
2202147992412,4cyclictest161rcu_preempt20:35:362
220214799241,12cyclictest2417220-21unixbench_singl22:40:452
220214799240,8cyclictest2503577-21sed00:10:342
220214799240,8cyclictest2498917-21tr00:05:362
220214799240,8cyclictest2468432-21sed23:35:312
220214799240,8cyclictest2274302-21sed20:20:332
220214799240,8cyclictest2209016-21tr19:15:052
220214799240,7cyclictest2244664-21sed19:50:332
220214799240,4cyclictest2478798-21kthreadcore23:45:332
220214799240,13cyclictest2284211-21cut20:30:332
220214199249,5cyclictest2453231-21cat23:20:270
220214199246,5cyclictest2217852-21ntpq19:20:380
220214799235,10cyclictest0-21swapper/222:45:352
220214799234,4cyclictest39-21ksoftirqd/220:55:342
220214799230,8cyclictest2485152-21cut23:50:362
220214799230,8cyclictest2202324-21cut19:10:272
220214199233,14cyclictest2202140-21wc19:10:010
2202141992318,4cyclictest2279184-21sort20:25:330
2202141992312,4cyclictest0-21swapper/023:50:360
220214799222,5cyclictest0-21swapper/223:20:312
22972042210,3sleep375-21kworker/3:1+events20:40:373
21977722214,10sleep20-21swapper/219:05:302
22004362183,8sleep30-21swapper/319:05:363
22680922160,2sleep3471rcuc/320:10:473
24927952144,7sleep30-21swapper/300:00:283
25222672132,5sleep30-21swapper/300:30:023
25183322132,7sleep30-21swapper/300:25:363
24626462132,6sleep30-21swapper/323:30:013
24432132132,6sleep30-21swapper/323:10:313
23600682133,6sleep30-21swapper/321:45:353
22427342132,7sleep30-21swapper/319:45:373
25321312122,6sleep30-21swapper/300:40:003
24884982122,6sleep30-21swapper/323:55:343
24854002122,6sleep30-21swapper/323:50:373
24714482122,6sleep30-21swapper/323:35:363
24395292122,6sleep30-21swapper/323:05:343
24369762122,6sleep30-21swapper/323:00:453
24268632122,6sleep30-21swapper/322:50:453
24204222122,6sleep30-21swapper/322:45:363
24076642122,6sleep30-21swapper/322:35:003
24076642122,6sleep30-21swapper/322:34:593
23878902122,6sleep30-21swapper/322:15:253
23778422122,6sleep30-21swapper/322:05:273
23717922122,6sleep30-21swapper/321:55:433
23656842122,6sleep30-21swapper/321:50:383
23531012122,6sleep30-21swapper/321:40:273
23440452123,5sleep30-21swapper/321:30:343
23377712122,5sleep30-21swapper/321:25:013
23330952122,6sleep30-21swapper/321:20:283
23323852122,6sleep30-21swapper/321:15:473
23251302123,5sleep30-21swapper/321:10:363
23087012123,6sleep30-21swapper/320:55:313
23038322122,6sleep30-21swapper/320:50:303
22989362122,6sleep30-21swapper/320:45:333
22835922122,6sleep30-21swapper/320:30:283
22784012122,6sleep30-21swapper/320:25:013
22740032122,6sleep30-21swapper/320:20:333
22326532122,6sleep30-21swapper/319:35:383
22141292122,6sleep30-21swapper/319:20:253
22135912122,6sleep30-21swapper/319:16:483
25125122112,5sleep30-21swapper/300:20:013
25122422112,6sleep30-21swapper/300:18:383
25068512112,5sleep30-21swapper/300:10:473
24978392112,5sleep30-21swapper/300:05:293
24814432112,5sleep30-21swapper/323:45:383
24769902112,5sleep30-21swapper/323:40:463
24584442112,6sleep30-21swapper/323:25:333
24518032112,5sleep30-21swapper/323:15:373
24279672112,5sleep30-21swapper/322:55:063
24128992112,5sleep30-21swapper/322:40:013
24117222111,6sleep30-21swapper/322:35:403
24117222111,6sleep30-21swapper/322:35:403
23975682112,5sleep30-21swapper/322:25:013
23932682112,5sleep30-21swapper/322:20:323
23831412112,6sleep30-21swapper/322:10:323
23478662112,5sleep30-21swapper/321:35:253
23228362112,6sleep30-21swapper/321:09:283
22886882112,5sleep30-21swapper/320:35:293
22712302112,5sleep30-21swapper/320:15:353
22626892112,6sleep30-21swapper/320:05:443
22536052112,5sleep30-21swapper/320:00:023
22385402112,5sleep30-21swapper/319:41:563
22385402112,5sleep30-21swapper/319:41:553
22242972112,6sleep30-21swapper/319:30:283
22190132112,5sleep30-21swapper/319:25:023
22020672112,5sleep30-21swapper/319:10:013
22439262102,5sleep30-21swapper/319:50:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional