You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-25 - 02:42

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Sat Apr 25, 2026 00:43:16)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2217057996254,5cyclictest2289210-21sh21:55:011
2217057996254,5cyclictest2289210-21sh21:55:011
2217057996051,6cyclictest2325490-21munin-run22:40:011
2217056995644,6cyclictest2247991-21sshd20:50:430
2217057995546,6cyclictest411063-21cups-browsed20:00:411
2217057995344,6cyclictest2406116-21idleruntime-cro00:20:011
2217057995142,5cyclictest2232040-21sendmail20:00:011
2217057995140,7cyclictest2299017-21/usr/sbin/munin22:05:531
2217056995130,11cyclictest151ktimers/022:45:500
2217056995130,11cyclictest151ktimers/022:45:500
221705799494,34cyclictest2302499-21ssh22:10:401
2217057994939,6cyclictest2224561-21systemd-run19:35:011
2217056994839,6cyclictest2386486-21aten_r2power_en23:55:320
2217056994720,19cyclictest12350irq/32-eth021:45:450
2217057994632,9cyclictest420-21avahi-daemon23:38:191
2217058994539,3cyclictest0-21swapper/200:00:012
2217057994434,6cyclictest421-21dbus-daemon20:40:011
2217057994434,6cyclictest2262610-21proc_pri21:20:461
2217057994427,12cyclictest420-21avahi-daemon21:15:471
2217058994337,4cyclictest2328332-21kworker/u8:2-rpciod22:50:362
2217058994336,4cyclictest2361782-21systemd-run23:25:022
2217058994336,4cyclictest2361782-21systemd-run23:25:022
2217058994335,5cyclictest2394029-21ls00:05:002
2217057994334,6cyclictest2420078-21unixbench_singl00:35:541
2217057994334,6cyclictest2420078-21unixbench_singl00:35:541
2217056994335,5cyclictest421-21dbus-daemon22:25:010
221705699431,40cyclictest2400902-21kworker/u8:12+rpciod00:25:440
2217058994136,2cyclictest0-21swapper/221:25:022
2217058994135,3cyclictest0-21swapper/200:15:422
2217058994133,5cyclictest2375692-21tail23:40:542
2217058994133,5cyclictest2232075-21latency_hist20:00:022
2217057994134,4cyclictest0-21swapper/100:05:421
2217057994133,3cyclictest0-21swapper/121:35:011
221705799413,30cyclictest411063-21cups-browsed21:25:411
2217057994129,8cyclictest421-21dbus-daemon00:30:361
221705799410,34cyclictest2399264-21ssh00:10:411
2217058994032,5cyclictest421-21dbus-daemon23:35:012
2217057994034,3cyclictest0-21swapper/120:45:491
2217057994033,3cyclictest0-21swapper/123:55:011
2217057994030,7cyclictest2351218-21ssh23:10:501
2217057994028,8cyclictest420-21avahi-daemon23:31:091
221705799402,26cyclictest2391404-21ssh00:00:481
2217056994033,4cyclictest2238069-21systemd-run20:20:010
2217056994031,6cyclictest421-21dbus-daemon23:20:530
2217056994031,6cyclictest421-21dbus-daemon23:20:530
2217056994031,5cyclictest2403711-21switchtime00:15:540
2217058993930,6cyclictest421-21dbus-daemon00:25:012
221705799393,26cyclictest2282443-21ssh21:45:461
2217057993931,5cyclictest2341584-21grep23:00:001
221705799392,28cyclictest2407431-21ssh00:20:451
2217056993929,7cyclictest2314042-21aten_r2power_po22:25:330
2217056993923,10cyclictest2354879-21ssh23:15:410
2217058993833,2cyclictest0-21swapper/200:05:402
2217058993830,4cyclictest0-21swapper/221:55:472
2217057993832,3cyclictest0-21swapper/100:00:011
2217057993829,6cyclictest411063-21cups-browsed23:20:531
2217057993829,6cyclictest411063-21cups-browsed23:20:531
2217057993829,6cyclictest2314092-21head22:25:331
221705799382,28cyclictest2278514-21ssh21:40:511
2217056993829,6cyclictest420-21avahi-daemon21:42:550
221705699382,24cyclictest2298555-21sh22:05:440
221705899374,27cyclictest2411392-21ssh00:25:422
2217058993730,3cyclictest0-21swapper/223:30:012
2217058993730,3cyclictest0-21swapper/223:10:362
2217058993729,5cyclictest421-21dbus-daemon19:15:012
2217058993729,5cyclictest420-21avahi-daemon20:25:022
2217058993728,6cyclictest1-21systemd22:25:012
2217057993729,5cyclictest2220730-21latency19:20:371
2217057993728,6cyclictest2235040-21idleruntime-cro20:10:011
2217057993727,6cyclictest421-21dbus-daemon22:45:021
2217057993727,6cyclictest421-21dbus-daemon22:45:021
2217056993730,5cyclictest2362389-21aten_r2power_vo23:25:330
2217056993730,4cyclictest2239495-21unixbench_singl20:20:520
2217058993629,4cyclictest1055-21mta-sts-daemon19:25:472
2217058993629,3cyclictest0-21swapper/223:15:422
2217058993628,5cyclictest420-21avahi-daemon19:45:492
2217058993627,6cyclictest2227571-21systemd-run19:45:012
2217058993627,6cyclictest2227571-21systemd-run19:45:012
2217058993626,7cyclictest1051-21mta-sts-daemon20:00:422
2217058993625,8cyclictest2370750-21kernelversion23:35:372
2217058993622,10cyclictest548-21gdbus23:46:352
221705799363,23cyclictest2362988-21ssh23:25:431
2217057993629,3cyclictest0-21swapper/120:20:011
2217057993628,4cyclictest0-21swapper/122:30:311
2217057993623,8cyclictest2297270-21cron22:05:001
2217057993612,18cyclictest411063-21cups-browsed23:00:411
2217056993630,4cyclictest2419445-21sshd00:35:420
2217056993630,4cyclictest2419445-21sshd00:35:420
2217056993628,5cyclictest2399940-21unixbench_singl00:10:540
2217056993624,5cyclictest2389957-21cups-browsed00:20:420
2217056993619,9cyclictest151ktimers/021:20:420
221705699360,16cyclictest2290453-21sshd21:55:430
221705899353,27cyclictest2286349-21ssh21:50:412
221705899353,27cyclictest2286349-21ssh21:50:412
2217058993529,3cyclictest0-21swapper/221:20:012
2217058993528,4cyclictest0-21swapper/219:10:022
2217058993528,3cyclictest0-21swapper/200:15:012
2217058993527,5cyclictest411063-21cups-browsed22:03:182
2217058993527,5cyclictest2226033-21munin-run19:40:012
2217058993527,5cyclictest1-21systemd22:45:022
2217058993527,5cyclictest1-21systemd22:45:022
2217058993524,8cyclictest421-21dbus-daemon20:15:442
221705799357,5cyclictest171rcu_preempt21:00:461
221705799357,5cyclictest171rcu_preempt21:00:461
2217057993528,4cyclictest1747350-21kworker/u8:2-xprtiod19:20:011
2217057993524,7cyclictest2330338-21fschecks_count22:45:371
2217057993524,7cyclictest2330338-21fschecks_count22:45:371
2217056993528,5cyclictest2395895-21unixbench_singl00:05:540
2217056993528,5cyclictest2350615-21kernelversion23:10:350
2217056993525,6cyclictest151ktimers/019:10:420
2217056993524,7cyclictest12450irq/33-eth023:05:430
2217058993427,4cyclictest2270142-21kernelversion21:30:382
2217058993426,5cyclictest2279080-21unixbench_singl21:41:012
2217058993426,3cyclictest0-21swapper/221:47:432
2217058993425,6cyclictest1056-21mta-sts-daemon19:35:002
221705799343,20cyclictest2290456-21ssh21:55:431
2217057993428,3cyclictest0-21swapper/123:20:011
2217057993426,5cyclictest2250079-21readlink21:00:001
2217057993426,5cyclictest2248589-21systemd-run20:55:011
2217057993425,5cyclictest2277206-21kworker/u8:4-xprtiod22:55:001
2217057993424,6cyclictest0-21swapper/121:40:011
221705799342,25cyclictest2306777-21ssh22:15:501
2217056993427,5cyclictest2219053-21fschecks_count19:15:350
2217056993426,5cyclictest2273348-21latency_hist21:35:020
2217056993412,15cyclictest151ktimers/019:55:480
2217058993327,3cyclictest0-21swapper/222:15:422
2217058993327,3cyclictest0-21swapper/221:35:462
2217058993326,4cyclictest1055-21mta-sts-daemon22:57:222
2217058993326,3cyclictest0-21swapper/222:45:062
2217058993326,3cyclictest0-21swapper/222:45:062
2217058993325,5cyclictest2299090-21unixbench_multi22:05:542
2217058993324,6cyclictest1056-21mta-sts-daemon00:35:312
2217058993324,6cyclictest1056-21mta-sts-daemon00:35:312
2217058993324,5cyclictest2343404-21grep23:00:542
221705699335,21cyclictest12350irq/32-eth022:00:430
221705699333,24cyclictest2366928-21ssh23:30:390
2217056993327,4cyclictest2253100-21systemd-run21:10:010
2217056993327,3cyclictest0-21swapper/021:20:000
2217056993326,4cyclictest0-21swapper/021:30:020
2217056993325,5cyclictest2273897-21sh21:35:350
2217058993227,2cyclictest0-21swapper/221:00:002
2217058993226,3cyclictest0-21swapper/223:50:422
2217058993224,6cyclictest0-21swapper/222:40:012
2217058993223,6cyclictest2244069-21cron20:40:012
221705799323,17cyclictest2379151-21ssh23:45:451
2217057993226,3cyclictest0-21swapper/121:10:011
2217057993225,4cyclictest2411366-21ssh00:25:421
2217057993224,5cyclictest2236032-21/usr/sbin/munin20:10:471
2217057993221,8cyclictest2346898-21awk23:05:431
2217057993216,6cyclictest0-21swapper/122:20:441
221705799321,22cyclictest2375050-21ssh23:40:411
2217056993226,3cyclictest0-21swapper/000:00:420
2217056993225,4cyclictest2382726-21gltestperf23:50:350
2217056993225,4cyclictest2371338-21mailstats23:35:520
2217056993225,4cyclictest0-21swapper/023:42:390
2217056993223,6cyclictest1054-21mta-sts-daemon21:10:040
2217056993214,11cyclictest12350irq/32-eth021:50:480
2217056993214,11cyclictest12350irq/32-eth021:50:480
2217058993125,3cyclictest0-21swapper/221:25:432
2217058993124,4cyclictest1055-21mta-sts-daemon19:20:012
2217058993122,7cyclictest2415149-21kernelversion00:30:372
221705899312,23cyclictest2346966-21ssh23:05:452
221705799314,17cyclictest2254405-21ssh21:10:441
2217057993122,6cyclictest1-21systemd19:40:011
2217057993121,6cyclictest2228058-21fschecks_count19:45:351
2217057993120,2cyclictest2240526-21ntpq20:25:511
2217056993125,4cyclictest2309370-21munin-run22:20:010
2217056993125,3cyclictest1055-21mta-sts-daemon20:40:510
2217056993122,7cyclictest151ktimers/022:35:460
2217058993024,4cyclictest2217055-21cyclictest21:10:312
2217058993024,3cyclictest0-21swapper/222:25:422
2217058993023,3cyclictest0-21swapper/221:10:012
2217058993020,7cyclictest2230524-21cron19:55:002
221705799302,21cyclictest411063-21cups-browsed20:30:471
2217056993023,5cyclictest2379218-21ntpq23:45:470
2217056993022,5cyclictest151ktimers/020:05:410
2217056993021,7cyclictest1055-21mta-sts-daemon20:10:310
221705699301,24cyclictest411055-21cupsd22:40:420
221705699301,24cyclictest411055-21cupsd22:40:420
2217058992922,3cyclictest0-21swapper/222:30:482
2217058992919,7cyclictest2248569-21idleruntime-cro20:55:002
2217056992924,3cyclictest2337556-21grep22:55:000
2217056992923,3cyclictest0-21swapper/022:30:370
2217056992922,4cyclictest516-21in:imuxsock22:15:010
2217056992922,4cyclictest2221161-21systemctl19:20:500
2217056992918,8cyclictest0-21swapper/000:30:550
2217058992823,2cyclictest0-21swapper/222:10:322
2217057992811,4cyclictest171rcu_preempt19:25:441
2217056992822,4cyclictest2339368-21tail22:55:520
2217056992821,4cyclictest2247019-21unixbench_singl20:45:550
2217056992815,10cyclictest1-21systemd20:55:430
2217058992719,5cyclictest411063-21cups-browsed20:32:142
2217058992719,5cyclictest411063-21cups-browsed20:09:082
221705799275,14cyclictest421-21dbus-daemon19:50:341
221705799274,16cyclictest428-21mta-sts-daemon19:40:411
221705799274,16cyclictest428-21mta-sts-daemon19:40:411
2217057992720,4cyclictest2217556-21fschecks_count19:10:331
2217057992718,5cyclictest2239433-21unixbench_multi20:20:511
221705699272,20cyclictest2229953-21sshd19:50:430
2217056992721,3cyclictest0-21swapper/023:00:520
2217056992720,5cyclictest431-21rsyslogd19:30:520
2217056992718,7cyclictest2227227-21switchtime19:40:520
2217056992718,7cyclictest2227227-21switchtime19:40:520
2217056992718,6cyclictest420-21avahi-daemon19:27:280
2217058992621,2cyclictest0-21swapper/220:25:392
2217057992618,5cyclictest2245623-21latency_hist20:45:021
2217056992618,6cyclictest695-21cron20:30:030
2217056992618,5cyclictest1056-21mta-sts-daemon20:00:430
2217056992618,4cyclictest0-21swapper/019:35:430
2217056992612,9cyclictest151ktimers/020:35:470
2217056992518,4cyclictest2251153-21mailstats21:00:480
2217056992518,4cyclictest2251153-21mailstats21:00:480
2217058992417,4cyclictest411063-21cups-browsed21:04:362
2217058992417,4cyclictest411063-21cups-browsed21:04:362
2217058992417,4cyclictest1051-21mta-sts-daemon20:45:522
2217058992416,5cyclictest2221545-21cron19:25:012
2217056992418,4cyclictest2239804-21aten_r2power_cu20:25:330
2217058992316,4cyclictest2245461-21unixbench_multi20:40:522
2217058992315,5cyclictest2236520-21cron20:15:012
2217057992215,3cyclictest0-21swapper/119:10:011
2217056992216,4cyclictest2228071-21fschecks_time19:45:350
221705999182,9cyclictest0-21swapper/323:25:483
221705999181,7cyclictest0-21swapper/321:15:433
221705999173,7cyclictest0-21swapper/323:15:433
221705999172,7cyclictest0-21swapper/300:15:423
221705999171,9cyclictest0-21swapper/323:05:443
2217056991713,2cyclictest0-21swapper/019:10:010
221705999165,7cyclictest0-21swapper/321:45:463
221705999162,8cyclictest0-21swapper/323:40:443
221705999162,8cyclictest0-21swapper/323:10:383
221705999162,7cyclictest0-21swapper/322:35:433
221705999162,7cyclictest0-21swapper/322:05:433
221705999152,8cyclictest0-21swapper/323:35:413
221705999152,8cyclictest0-21swapper/322:00:423
221705999152,7cyclictest0-21swapper/322:50:413
221705999152,7cyclictest0-21swapper/321:35:453
221705999152,7cyclictest0-21swapper/321:30:423
221705999151,8cyclictest0-21swapper/321:55:413
221705999151,7cyclictest0-21swapper/320:00:433
221705999151,7cyclictest0-21swapper/319:15:413
221705999142,9cyclictest0-21swapper/321:25:413
221705999142,7cyclictest0-21swapper/323:50:413
221705999142,7cyclictest0-21swapper/322:56:393
221705999142,7cyclictest0-21swapper/300:20:423
221705999142,7cyclictest0-21swapper/300:05:413
221705999142,6cyclictest0-21swapper/321:40:463
221705999142,5cyclictest0-21swapper/320:55:403
221705999141,8cyclictest0-21swapper/323:30:393
221705999141,8cyclictest0-21swapper/321:10:413
221705999141,8cyclictest0-21swapper/300:00:453
221705999141,7cyclictest0-21swapper/322:45:453
221705999141,7cyclictest0-21swapper/322:45:453
221705999141,7cyclictest0-21swapper/321:20:433
221705999141,7cyclictest0-21swapper/319:25:423
221705999141,6cyclictest0-21swapper/321:05:403
221705999141,6cyclictest0-21swapper/320:20:403
221705999141,5cyclictest0-21swapper/319:40:403
221705999141,5cyclictest0-21swapper/319:40:403
221705999134,6cyclictest0-21swapper/323:55:433
221705999132,7cyclictest0-21swapper/322:40:453
221705999132,7cyclictest0-21swapper/322:40:453
221705999132,7cyclictest0-21swapper/322:20:433
221705999132,7cyclictest0-21swapper/322:15:413
221705999132,7cyclictest0-21swapper/319:55:013
221705999131,8cyclictest0-21swapper/322:25:443
221705999131,8cyclictest0-21swapper/300:10:413
221705999131,7cyclictest0-21swapper/323:00:443
221705999131,7cyclictest0-21swapper/322:10:413
221705999131,7cyclictest0-21swapper/300:35:423
221705999131,7cyclictest0-21swapper/300:35:423
221705999131,7cyclictest0-21swapper/300:30:453
221705999131,6cyclictest0-21swapper/321:00:463
221705999131,6cyclictest0-21swapper/321:00:463
221705999131,6cyclictest0-21swapper/320:35:423
221705999131,6cyclictest0-21swapper/320:05:403
221705999131,5cyclictest0-21swapper/320:45:413
221705999123,4cyclictest0-21swapper/320:25:513
221705999122,7cyclictest0-21swapper/323:45:423
221705999122,6cyclictest0-21swapper/322:30:433
221705999122,4cyclictest0-21swapper/320:10:423
221705999122,4cyclictest0-21swapper/319:10:393
221705999121,7cyclictest0-21swapper/323:20:413
221705999121,7cyclictest0-21swapper/323:20:413
221705999121,7cyclictest0-21swapper/321:50:403
221705999121,7cyclictest0-21swapper/321:50:403
221705999121,6cyclictest0-21swapper/320:30:453
221705999114,5cyclictest0-21swapper/320:15:413
221705999111,7cyclictest0-21swapper/300:25:373
221705999111,6cyclictest0-21swapper/319:30:413
221705999111,5cyclictest0-21swapper/320:50:413
221705999111,5cyclictest0-21swapper/319:35:403
221705999110,6cyclictest0-21swapper/319:55:443
221705999102,4cyclictest0-21swapper/319:45:363
221705999101,5cyclictest0-21swapper/319:20:333
22170599992,4cyclictest0-21swapper/320:40:323
22170599991,5cyclictest0-21swapper/319:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional