You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-10 - 19:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Fri Apr 10, 2026 12:43:25)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2322430997461,5cyclictest0-21swapper/011:50:370
2322430997143,20cyclictest12350irq/32-eth009:25:000
2322430996651,12cyclictest2520976-21sshd12:30:270
2322430995631,22cyclictest12350irq/32-eth010:20:340
2322431995142,6cyclictest2370740-21dpkg09:25:011
2322432994839,6cyclictest439-21systemd-logind09:05:012
2322432994839,6cyclictest439-21systemd-logind09:05:012
2322430994834,12cyclictest2431447-21munin-run10:40:010
2322430994823,3cyclictest0-21swapper/011:25:380
232243099473,18cyclictest2359769-21ps09:10:400
2322430994721,7cyclictest261-21systemd-journal12:35:280
2322430994717,23cyclictest2493375-21ssh11:55:340
2322432994635,7cyclictest1461217-21cups-browsed12:38:082
2322431994638,5cyclictest1054-21mta-sts-daemon08:20:341
2322431994636,6cyclictest0-21swapper/107:55:021
2322430994625,4cyclictest1056-21mta-sts-daemon10:40:040
2322431994537,5cyclictest2492936-21/usr/sbin/munin11:55:251
2322431994536,5cyclictest2427381-21cron10:35:011
232243199451,35cyclictest2359542-21ssh09:10:311
2322430994538,4cyclictest2334434-21kworker/u8:8-rpciod08:49:030
2322430994419,6cyclictest12350irq/32-eth010:15:310
2322432994332,7cyclictest420-21avahi-daemon09:20:132
2322432994325,4cyclictest171rcu_preempt12:15:382
2322432994229,10cyclictest2366610-21sh09:20:012
2322431994234,5cyclictest2322453-21latency_hist07:10:011
2322431994232,6cyclictest421-21dbus-daemon11:25:261
2322431994230,9cyclictest2392482-21xargs09:50:441
2322431994230,9cyclictest2392482-21xargs09:50:441
232243099423,29cyclictest2452868-21ssh11:05:340
232243099423,29cyclictest2452868-21ssh11:05:340
2322430994230,9cyclictest2388620-21unixbench_multi09:45:440
2322430994230,5cyclictest0-21swapper/010:45:390
2322432994132,6cyclictest1461217-21cups-browsed09:31:352
2322432994129,7cyclictest0-21swapper/210:05:312
2322431994132,6cyclictest2509893-21switchtime12:15:451
2322431994119,14cyclictest421-21dbus-daemon10:10:381
2322430994134,5cyclictest2399044-21systemd-run10:00:010
2322430994129,9cyclictest2482740-21sshd11:42:250
2322430994129,9cyclictest2482740-21sshd11:42:250
2322432994034,3cyclictest0-21swapper/211:40:232
2322432994034,3cyclictest0-21swapper/211:40:232
2322432994032,4cyclictest0-21swapper/211:35:402
2322432994031,6cyclictest420-21avahi-daemon07:35:252
2322432994029,7cyclictest420-21avahi-daemon10:35:562
2322432994029,3cyclictest0-21swapper/212:10:342
2322432994027,9cyclictest2408230-21ntp_kernel_err10:10:322
2322431994032,5cyclictest2439570-21systemd-run10:50:011
2322431994030,6cyclictest2435548-21latency_hist10:45:021
232243299394,32cyclictest2372013-21ssh09:25:402
2322432993931,5cyclictest2447632-21cron11:00:012
2322432993931,5cyclictest2355416-21systemctl09:00:002
2322432993931,5cyclictest2355416-21systemctl09:00:002
2322432993929,6cyclictest420-21avahi-daemon11:54:152
2322432993928,5cyclictest0-21swapper/208:05:332
2322432993925,11cyclictest2452782-21needreboot11:05:322
2322432993925,11cyclictest2452782-21needreboot11:05:322
2322431993932,3cyclictest0-21swapper/110:30:011
2322431993932,3cyclictest0-21swapper/110:30:011
2322431993932,3cyclictest0-21swapper/107:35:021
2322431993930,6cyclictest2354435-21fschecks_count08:55:271
2322431993930,6cyclictest2354435-21fschecks_count08:55:271
2322431993930,6cyclictest2322955-21fschecks_time07:10:261
2322431993930,6cyclictest2322955-21fschecks_time07:10:261
232243199392,27cyclictest2440748-21ssh10:50:351
2322430993928,6cyclictest151ktimers/007:40:350
232243299384,29cyclictest2469228-21ssh11:25:432
2322432993831,3cyclictest0-21swapper/210:45:392
2322432993830,5cyclictest2518042-21tail12:25:482
2322432993830,5cyclictest2518042-21tail12:25:482
2322432993830,5cyclictest2387595-21grep09:45:262
2322432993829,4cyclictest0-21swapper/210:45:012
232243299381,29cyclictest2412403-21ssh10:15:352
2322431993832,3cyclictest0-21swapper/111:50:251
2322431993830,5cyclictest2488226-21latency_hist11:50:011
2322431993830,5cyclictest2488226-21latency_hist11:50:011
2322431993829,5cyclictest2334433-21kworker/u8:4-xprtiod08:40:011
2322431993828,5cyclictest0-21swapper/110:55:401
2322431993827,7cyclictest2364272-21unixbench_multi09:15:461
2322431993811,22cyclictest2505588-21sh12:10:371
2322430993826,9cyclictest0-21swapper/010:25:450
2322430993826,9cyclictest0-21swapper/010:25:450
2322432993728,6cyclictest2400779-21unixbench_multi10:00:462
2322432993728,5cyclictest2493902-21unixbench_multi11:55:432
2322432993728,3cyclictest0-21swapper/210:20:382
232243199373,28cyclictest2513546-21ssh12:20:311
232243199373,28cyclictest2513546-21ssh12:20:311
2322431993731,3cyclictest0-21swapper/112:10:011
2322431993731,3cyclictest0-21swapper/109:55:471
2322431993729,5cyclictest2525966-21unixbench_multi12:35:451
2322431993727,7cyclictest2403933-21if_wlan010:05:271
2322431993724,10cyclictest2358417-21diskmemload09:25:341
2322430993731,4cyclictest2512422-21ssh12:20:010
2322430993728,6cyclictest151ktimers/011:00:390
2322430993728,6cyclictest151ktimers/011:00:390
2322430993725,9cyclictest439-21systemd-logind10:35:010
232243299365,22cyclictest2513783-21ssh12:20:392
232243299365,22cyclictest2513783-21ssh12:20:392
232243299364,24cyclictest2472614-21ssh11:30:252
2322432993628,5cyclictest2399044-21systemd-run10:00:012
2322432993628,5cyclictest2359876-21mailstats09:10:412
2322432993627,4cyclictest0-21swapper/211:15:432
2322432993626,7cyclictest2423489-21latency_hist10:30:022
2322432993626,7cyclictest2423489-21latency_hist10:30:022
232243199365,23cyclictest2384026-21ssh09:40:351
232243199364,26cyclictest2481153-21ssh11:40:311
232243199364,26cyclictest2481153-21ssh11:40:311
232243199363,25cyclictest2521368-21ssh12:30:331
2322431993628,5cyclictest2456279-21aten_r2power_po11:10:241
2322431993628,5cyclictest2387596-21fschecks_time09:45:261
232243199361,30cyclictest2517403-21sh12:25:341
232243199361,30cyclictest2517403-21sh12:25:341
2322430993630,4cyclictest421-21dbus-daemon08:20:010
2322430993630,4cyclictest2350916-21cron08:45:010
2322430993629,4cyclictest2400757-21unixbench-2d10:00:450
232243299353,23cyclictest2497608-21ssh12:00:422
2322432993529,3cyclictest2329930-21kworker/u8:1-nfsiod08:49:022
2322432993528,3cyclictest0-21swapper/208:20:012
2322432993525,6cyclictest1461210-21cupsd11:48:562
2322432993525,6cyclictest1461210-21cupsd11:48:562
232243299352,22cyclictest1461217-21cups-browsed07:40:332
2322431993527,5cyclictest2461571-21unixbench_singl11:15:471
2322431993527,5cyclictest2343430-21sendmail08:20:011
2322431993526,6cyclictest1461217-21cups-browsed11:32:361
2322431993523,9cyclictest2379399-21cpu09:35:251
232243199352,26cyclictest2428669-21ssh10:35:391
2322431993521,10cyclictest2480081-21sh11:40:011
2322430993529,4cyclictest2363153-21cpu09:15:250
2322430993526,6cyclictest1461217-21cups-browsed10:50:580
2322430993525,7cyclictest2484969-21kernelversion11:45:290
2322430993525,7cyclictest2484969-21kernelversion11:45:290
2322430993525,7cyclictest2333738-21needreboot07:45:300
2322430993523,9cyclictest0-21swapper/011:25:010
232243299343,23cyclictest2465028-21ssh11:20:362
2322432993424,7cyclictest2347157-21missed_timers08:30:292
232243299341,25cyclictest2392091-21ssh09:50:342
232243299341,25cyclictest2392091-21ssh09:50:342
232243199343,27cyclictest2416427-21sshd10:20:341
2322431993427,3cyclictest0-21swapper/110:00:371
2322431993426,5cyclictest421-21dbus-daemon11:06:441
2322431993426,5cyclictest421-21dbus-daemon11:06:441
232243199342,24cyclictest2375941-21ssh09:30:361
2322430993425,4cyclictest2439572-21kworker/u8:0-rpciod11:10:380
2322430993425,4cyclictest12450irq/33-eth012:10:370
2322430993415,14cyclictest12350irq/32-eth011:35:330
2322430993413,14cyclictest420-21avahi-daemon09:40:340
2322430993411,16cyclictest2497486-21ssh12:00:350
2322432993325,5cyclictest2449415-21unixbench_multi11:00:452
2322432993325,5cyclictest2449415-21unixbench_multi11:00:452
2322432993324,3cyclictest0-21swapper/212:30:272
2322432993323,7cyclictest2501806-21switchtime12:05:432
232243199333,24cyclictest2412362-21ssh10:15:341
2322431993326,4cyclictest0-21swapper/107:40:011
2322431993325,5cyclictest2340356-21unixbench_singl08:05:441
2322431993323,7cyclictest2337407-21grep08:00:011
2322431993322,6cyclictest0-21swapper/112:00:301
232243099334,24cyclictest2404358-21ssh10:05:370
2322430993325,6cyclictest2513494-21needreboot12:20:300
2322430993325,6cyclictest2513494-21needreboot12:20:300
2322430993325,5cyclictest2460752-21missed_timers11:15:310
2322430993317,14cyclictest2391919-21missed_timers09:50:290
2322430993317,14cyclictest2391919-21missed_timers09:50:290
232243299327,16cyclictest428-21mta-sts-daemon08:10:332
232243299323,22cyclictest2456959-21ssh11:10:372
2322432993226,3cyclictest0-21swapper/208:00:012
2322432993225,3cyclictest0-21swapper/207:15:012
2322432993225,3cyclictest0-21swapper/207:15:012
2322432993223,6cyclictest1-21systemd08:55:012
2322432993223,6cyclictest1-21systemd08:55:012
2322432993223,5cyclictest2427396-21grep10:35:012
2322432993223,5cyclictest1055-21mta-sts-daemon09:35:362
2322431993226,3cyclictest0-21swapper/108:49:031
2322431993226,3cyclictest0-21swapper/107:20:001
2322431993226,3cyclictest0-21swapper/107:20:001
2322431993225,5cyclictest2465522-21unixbench-2d11:20:451
2322431993225,4cyclictest1051-21mta-sts-daemon08:10:341
2322430993226,4cyclictest2349360-21head08:35:450
2322430993226,4cyclictest1785404-21kworker/u8:6-writeback07:15:360
2322430993226,4cyclictest1785404-21kworker/u8:6-writeback07:15:360
2322430993223,7cyclictest2345458-21fschecks_time08:25:290
2322431993125,3cyclictest0-21swapper/111:05:031
2322431993125,3cyclictest0-21swapper/111:05:031
2322430993125,3cyclictest0-21swapper/009:35:420
2322430993124,4cyclictest0-21swapper/009:25:420
232243099312,25cyclictest2517548-21ssh12:25:390
232243099312,25cyclictest2517548-21ssh12:25:390
232243099312,24cyclictest2408287-21ssh10:10:330
2322430993121,7cyclictest2323232-21needreboot07:10:300
2322430993121,7cyclictest2323232-21needreboot07:10:300
232243099311,15cyclictest2467878-21kworker/u8:12+rpciod11:30:320
2322432993023,3cyclictest0-21swapper/209:40:472
2322432993023,3cyclictest0-21swapper/208:40:012
2322432993020,7cyclictest1055-21mta-sts-daemon07:25:342
232243299301,21cyclictest2440669-21ssh10:50:332
232243199304,14cyclictest1054-21mta-sts-daemon08:50:341
232243199304,14cyclictest1054-21mta-sts-daemon08:50:341
2322430993023,3cyclictest0-21swapper/010:55:360
2322430993018,3cyclictest0-21swapper/008:00:010
2322430993010,16cyclictest2376033-21ssh09:30:400
2322432992920,6cyclictest2334026-21mailstats07:45:412
2322431992922,4cyclictest2347488-21sendmail_mailst08:30:391
2322431992922,4cyclictest1055-21mta-sts-daemon08:40:341
2322431992920,6cyclictest0-21swapper/109:10:011
2322431992920,6cyclictest0-21swapper/109:10:011
232243099292,23cyclictest659-21snmpd07:28:250
2322432992823,2cyclictest0-21swapper/207:20:002
2322432992823,2cyclictest0-21swapper/207:20:002
2322432992818,7cyclictest2335796-21grep07:50:452
232243199282,19cyclictest547-21gmain08:00:361
2322431992821,4cyclictest2346330-21unixbench_multi08:25:471
232243099282,21cyclictest1461217-21cups-browsed08:20:350
2322430992818,7cyclictest2323927-21kworker/u8:14-xprtiod07:30:280
2322431992718,6cyclictest2334347-21unixbench_singl07:45:441
2322430992722,3cyclictest0-21swapper/009:05:000
2322430992722,3cyclictest0-21swapper/009:05:000
2322430992721,4cyclictest2334948-21fschecks_time07:50:270
2322430992717,5cyclictest261-21systemd-journal09:00:000
2322430992717,5cyclictest261-21systemd-journal09:00:000
2322430992716,8cyclictest0-21swapper/008:00:250
2322430992714,7cyclictest0-21swapper/008:15:010
232243299262,19cyclictest600-21gmain08:00:332
2322432992620,3cyclictest0-21swapper/207:25:012
2322432992619,4cyclictest0-21swapper/208:25:012
2322432992618,5cyclictest2329909-21cron07:35:012
2322431992617,6cyclictest2355903-21/usr/sbin/munin09:00:251
2322431992617,6cyclictest2355903-21/usr/sbin/munin09:00:251
2322430992622,3cyclictest2322429-21cyclictest07:35:340
2322430992619,4cyclictest0-21swapper/008:05:330
2322430992610,12cyclictest2501695-21tr12:05:390
232243299257,5cyclictest171rcu_preempt08:40:342
2322431992517,5cyclictest1461217-21cups-browsed07:23:411
2322432992413,7cyclictest2358423-21basename09:10:002
2322432992413,7cyclictest2358423-21basename09:10:002
2322431992416,5cyclictest1461217-21cups-browsed07:26:371
2322430992316,5cyclictest2347919-21kworker/u8:12-rpciod09:10:020
2322430992316,5cyclictest2347919-21kworker/u8:12-rpciod09:10:020
2322430992316,2cyclictest0-21swapper/008:50:340
2322430992316,2cyclictest0-21swapper/008:50:340
2322432992216,3cyclictest0-21swapper/208:25:332
2322432992116,2cyclictest0-21swapper/207:10:012
2322431992115,3cyclictest0-21swapper/107:40:451
232243099212,14cyclictest2326393-21ntp_states07:20:380
232243099211,8cyclictest2347223-21needreboot08:30:290
2322430992014,3cyclictest0-21swapper/007:10:010
232243399172,7cyclictest0-21swapper/312:05:363
232243399164,5cyclictest0-21swapper/309:25:003
232243399164,4cyclictest0-21swapper/310:45:343
232243399162,8cyclictest0-21swapper/311:00:373
232243399162,8cyclictest0-21swapper/311:00:373
232243399162,8cyclictest0-21swapper/310:20:343
232243399162,8cyclictest0-21swapper/309:55:393
232243399158,5cyclictest0-21swapper/312:30:303
232243399155,7cyclictest0-21swapper/310:15:343
232243399154,6cyclictest0-21swapper/311:45:383
232243399154,6cyclictest0-21swapper/311:45:383
232243399153,5cyclictest0-21swapper/310:05:313
232243399153,5cyclictest0-21swapper/308:45:343
232243399152,8cyclictest0-21swapper/309:40:393
232243399152,7cyclictest0-21swapper/312:10:393
232243399151,8cyclictest0-21swapper/309:25:363
232243399151,7cyclictest0-21swapper/311:55:303
232243399151,7cyclictest0-21swapper/310:35:343
232243399151,7cyclictest0-21swapper/307:25:383
232243399151,6cyclictest0-21swapper/307:40:353
232243399144,5cyclictest0-21swapper/311:25:383
232243399143,6cyclictest0-21swapper/312:35:353
232243399143,6cyclictest0-21swapper/310:30:313
232243399143,5cyclictest0-21swapper/309:10:023
232243399143,5cyclictest0-21swapper/309:10:023
232243399143,4cyclictest0-21swapper/307:15:323
232243399143,4cyclictest0-21swapper/307:15:323
232243399142,8cyclictest0-21swapper/311:35:363
232243399142,8cyclictest0-21swapper/310:00:393
232243399142,7cyclictest0-21swapper/310:55:343
232243399142,7cyclictest0-21swapper/310:25:253
232243399142,7cyclictest0-21swapper/310:25:253
232243399142,6cyclictest0-21swapper/312:25:393
232243399142,6cyclictest0-21swapper/312:25:393
232243399142,6cyclictest0-21swapper/311:50:323
232243399141,8cyclictest0-21swapper/310:40:373
232243399141,8cyclictest0-21swapper/309:45:323
232243399141,7cyclictest0-21swapper/309:15:273
232243399141,6cyclictest0-21swapper/311:40:253
232243399141,6cyclictest0-21swapper/311:40:253
232243399141,6cyclictest0-21swapper/309:50:253
232243399141,6cyclictest0-21swapper/309:50:253
232243399141,6cyclictest0-21swapper/308:55:393
232243399141,6cyclictest0-21swapper/308:55:393
232243399141,6cyclictest0-21swapper/307:50:363
232243399141,6cyclictest0-21swapper/307:10:353
232243399141,6cyclictest0-21swapper/307:10:353
232243399136,4cyclictest0-21swapper/312:00:303
232243399132,7cyclictest0-21swapper/311:23:363
232243399132,7cyclictest0-21swapper/311:05:483
232243399132,7cyclictest0-21swapper/311:05:483
232243399132,7cyclictest0-21swapper/310:10:383
232243399132,6cyclictest0-21swapper/310:50:263
232243399131,7cyclictest0-21swapper/312:15:353
232243399131,7cyclictest0-21swapper/311:10:323
232243399131,6cyclictest0-21swapper/308:15:363
232243399131,6cyclictest0-21swapper/307:30:303
232243399131,5cyclictest0-21swapper/308:05:313
232243399123,4cyclictest0-21swapper/308:00:363
232243399122,6cyclictest0-21swapper/312:20:223
232243399122,6cyclictest0-21swapper/312:20:223
232243399121,7cyclictest0-21swapper/309:35:293
232243399121,7cyclictest0-21swapper/309:30:343
232243399116,3cyclictest0-21swapper/309:10:343
232243399115,4cyclictest0-21swapper/308:00:013
232243399111,7cyclictest0-21swapper/311:15:293
232243399111,5cyclictest0-21swapper/308:40:253
232243399103,4cyclictest0-21swapper/309:05:013
232243399103,4cyclictest0-21swapper/309:05:013
232243399103,4cyclictest0-21swapper/308:35:443
232243399101,5cyclictest0-21swapper/311:30:253
232243399101,5cyclictest0-21swapper/308:20:243
232243399101,5cyclictest0-21swapper/307:45:253
232243399101,5cyclictest0-21swapper/307:20:243
23224339994,3cyclictest0-21swapper/308:10:403
23224339992,5cyclictest0-21swapper/308:25:383
23224339984,3cyclictest0-21swapper/308:30:033
23224339982,5cyclictest0-21swapper/308:55:013
23224339982,5cyclictest0-21swapper/308:55:013
23224339981,5cyclictest0-21swapper/307:40:013
23224339981,5cyclictest0-21swapper/307:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional