You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-21 - 01:47

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Tue Jul 21, 2026 00:43:13)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
13848599649,7cyclictest171rcu_preempt21:50:491
138486994739,5cyclictest268504-21latency_hist23:45:012
138486994738,6cyclictest439-21systemd-logind20:55:022
138485994741,3cyclictest0-21swapper/123:30:011
138485994636,6cyclictest235264-21latency_hist22:50:021
13848499463,38cyclictest190885-21ssh21:35:470
138486994537,5cyclictest268995-21aten_r2power_fr23:45:352
138486994536,4cyclictest166949-21kworker/u8:0-xprtiod21:12:452
138486994535,6cyclictest253395-21latency_hist23:20:022
138486994523,19cyclictest171320-21unixbench_multi20:55:572
138485994535,7cyclictest2602812-21cupsd22:50:081
138486994436,5cyclictest214222-21latency_hist22:15:032
138486994419,2cyclictest0-21swapper/222:35:472
138486994417,3cyclictest0-21swapper/222:40:562
138485994434,6cyclictest300546-21unixbench_singl00:35:561
138485994431,8cyclictest277787-21cups-browsed00:14:561
138485994431,8cyclictest214457-21tail22:15:331
13848499442,35cyclictest187880-21ssh21:30:510
138486994337,3cyclictest0-21swapper/221:20:002
138486994337,3cyclictest0-21swapper/221:20:002
138486994335,5cyclictest277620-21sendmail00:00:012
138486994323,17cyclictest0-21swapper/221:31:002
13848699432,22cyclictest200006-21ssh21:50:462
138486994319,2cyclictest0-21swapper/219:20:012
138486994317,23cyclictest163584-21mailstats20:30:532
13848599437,29cyclictest193955-21ssh21:40:481
138485994328,12cyclictest291262-21sh00:20:581
138486994232,7cyclictest291401-21sed00:20:582
138486994220,18cyclictest204910-21munin-plugin-st22:00:012
138486994218,3cyclictest0-21swapper/223:30:022
138485994233,5cyclictest1003-21mta-sts-daemon23:42:461
138485994231,7cyclictest0-21swapper/100:00:021
138484994235,4cyclictest181306-21uniq21:20:380
138484994226,10cyclictest12650irq/32-eth023:15:490
138486994133,5cyclictest241252-21ssh23:00:012
138486994133,5cyclictest138495-21systemd-run19:10:012
138486994116,2cyclictest0-21swapper/200:05:532
138486994113,4cyclictest144867-21aten_r2power_vo19:30:342
138486994113,4cyclictest144867-21aten_r2power_vo19:30:342
138485994135,3cyclictest0-21swapper/123:45:421
138484994121,8cyclictest277597-21munin-run00:00:010
138486994032,5cyclictest198881-21kworker/u8:3-rpciod22:20:012
138486994020,17cyclictest1003-21mta-sts-daemon22:03:422
138486994017,4cyclictest1000-21mta-sts-daemon22:12:012
138486994017,3cyclictest0-21swapper/223:20:562
138485994033,4cyclictest232203-21munin-run22:45:011
138485994032,5cyclictest180583-21systemctl21:20:001
138485994032,5cyclictest180583-21systemctl21:20:001
138485994032,5cyclictest145969-21munin-run19:35:001
138485994032,5cyclictest145969-21munin-run19:35:001
138485994032,4cyclictest0-21swapper/123:15:011
138485994032,4cyclictest0-21swapper/123:15:011
138485994027,9cyclictest420-21avahi-daemon22:33:131
138484994031,6cyclictest1-21systemd22:30:020
13848499402,31cyclictest193888-21ssh21:40:450
13848699393,10cyclictest428-21mta-sts-daemon20:20:482
138486993929,7cyclictest186689-21sh21:30:012
138486993927,7cyclictest0-21swapper/200:15:502
138486993921,15cyclictest0-21swapper/223:36:012
138486993917,5cyclictest2602812-21cupsd20:40:532
138486993917,5cyclictest2602812-21cupsd20:40:532
138486993916,3cyclictest0-21swapper/221:05:552
138486993916,2cyclictest0-21swapper/222:30:452
138486993915,3cyclictest0-21swapper/223:50:562
138486993914,6cyclictest232762-21cut22:45:352
138486993912,5cyclictest154981-21munin-plugin-st20:05:012
13848699391,14cyclictest294047-21ssh00:25:512
138485993930,6cyclictest0-21swapper/123:20:011
138485993930,5cyclictest260116-21fschecks_time23:30:371
138485993928,5cyclictest0-21swapper/122:00:521
138485993926,10cyclictest161848-21ntp_kernel_pll_20:25:431
13848699384,12cyclictest196970-21ssh21:45:452
138486993829,5cyclictest0-21swapper/200:40:012
13848599383,27cyclictest2602816-21cups-browsed20:00:451
138485993827,7cyclictest221419-21sh22:25:541
138485993823,10cyclictest2602816-21cups-browsed22:09:431
13848599382,23cyclictest665-21snmpd21:20:481
138484993832,4cyclictest241252-21ssh23:00:010
138484993832,3cyclictest0-21swapper/023:45:010
138484993830,5cyclictest420-21avahi-daemon23:32:200
138484993829,7cyclictest226150-21systemd22:35:020
138484993827,8cyclictest153461-21munin-plugin-st20:00:010
138484993827,6cyclictest151ktimers/000:10:480
138484993825,10cyclictest151954-21cron19:55:010
138486993728,7cyclictest244321-21systemd-run23:05:022
13848699372,6cyclictest193859-21ntpq21:40:442
13848599373,27cyclictest187770-21ssh21:30:451
138485993729,5cyclictest280655-21ssh00:05:011
138485993729,5cyclictest254984-21grep23:20:571
138485993728,4cyclictest0-21swapper/121:28:041
138485993725,9cyclictest1002-21mta-sts-daemon00:17:221
138485993724,9cyclictest1-21systemd00:30:351
138484993729,5cyclictest230833-21unixbench_multi22:40:570
138484993728,6cyclictest151ktimers/022:10:500
138484993728,6cyclictest151ktimers/022:10:500
138484993725,7cyclictest293931-21ssh00:25:460
138486993630,3cyclictest159490-21sendmail20:20:012
138486993629,4cyclictest286717-21idleruntime-cro00:15:012
138486993616,17cyclictest0-21swapper/219:10:492
138486993616,14cyclictest288804-21kworker/u8:9+xprtiod00:30:442
138486993614,3cyclictest0-21swapper/222:25:022
138486993613,5cyclictest0-21swapper/222:25:502
13848699361,14cyclictest190937-21ssh21:35:512
13848699361,12cyclictest248423-21ssh23:10:452
13848699361,12cyclictest248423-21ssh23:10:452
138485993630,3cyclictest0-21swapper/121:00:011
138485993628,6cyclictest138483-21cyclictest20:40:481
138485993628,6cyclictest138483-21cyclictest20:40:481
138485993628,4cyclictest1000-21mta-sts-daemon22:12:071
138485993628,4cyclictest1000-21mta-sts-daemon22:12:071
138485993626,6cyclictest281682-21needreboot00:05:401
138484993631,3cyclictest147375-21grep19:35:560
138484993631,3cyclictest147375-21grep19:35:560
138484993630,4cyclictest250308-21munin-run23:15:000
138484993630,4cyclictest250308-21munin-run23:15:000
138484993630,4cyclictest217029-21munin-run22:20:010
138484993630,4cyclictest180573-21sendmail21:20:000
138484993630,4cyclictest180573-21sendmail21:20:000
138484993628,5cyclictest0-21swapper/021:50:020
138484993626,7cyclictest271927-21/usr/sbin/munin23:50:400
138484993626,7cyclictest245251-21missed_timers23:05:420
138484993626,7cyclictest245251-21missed_timers23:05:420
138484993625,7cyclictest151ktimers/020:40:450
138484993625,7cyclictest151ktimers/020:40:450
138484993622,11cyclictest278527-21latency00:00:390
138484993620,11cyclictest0-21swapper/022:35:510
138486993528,5cyclictest0-21swapper/220:15:012
13848699352,13cyclictest165013-21ps20:35:522
138486993514,2cyclictest0-21swapper/200:02:252
13848699351,14cyclictest236359-21ntpq22:50:462
13848699351,13cyclictest260336-21memory23:30:402
138485993527,5cyclictest0-21swapper/120:20:011
138485993523,6cyclictest0-21swapper/123:04:391
13848499353,24cyclictest287827-21ssh00:15:450
138484993529,4cyclictest1001-21mta-sts-daemon22:48:470
138484993529,3cyclictest0-21swapper/020:20:450
138484993528,4cyclictest290618-21kernelversion00:20:410
138484993528,4cyclictest148980-21grep19:45:000
138484993527,4cyclictest151ktimers/020:05:450
138486993425,5cyclictest0-21swapper/219:55:432
138486993414,4cyclictest1001-21mta-sts-daemon21:23:572
138486993413,2cyclictest0-21swapper/219:40:342
138485993428,3cyclictest0-21swapper/121:05:011
138485993426,5cyclictest144543-21latency_hist19:30:021
138485993424,6cyclictest241441-21latency_hist23:00:031
138485993424,5cyclictest0-21swapper/122:35:391
138485993423,5cyclictest0-21swapper/100:25:521
13848599342,25cyclictest272663-21ssh23:50:471
138484993427,4cyclictest140478-21fschecks_count19:15:380
138484993427,4cyclictest1002-21mta-sts-daemon21:28:240
138484993426,5cyclictest0-21swapper/021:15:010
13848699333,24cyclictest245367-21ssh23:05:452
13848699333,24cyclictest245367-21ssh23:05:452
138486993326,4cyclictest149508-21fschecks_time19:45:372
138485993328,2cyclictest0-21swapper/120:00:011
138485993327,3cyclictest0-21swapper/122:00:011
138485993325,5cyclictest1003-21mta-sts-daemon23:05:461
138485993325,5cyclictest1003-21mta-sts-daemon23:05:461
138485993324,6cyclictest150987-21fschecks_count19:50:381
138485993322,6cyclictest0-21swapper/121:10:491
138484993325,6cyclictest210987-21sh22:10:010
138484993314,14cyclictest627-21lldpd20:25:420
138484993314,13cyclictest151ktimers/000:35:480
138485993224,5cyclictest190852-21ssh21:35:461
138485993224,5cyclictest147462-21munin-run19:40:001
138485993224,5cyclictest147462-21munin-run19:40:001
138485993221,6cyclictest218722-21unixbench_multi22:20:571
138484993226,4cyclictest154993-21systemd-run20:05:010
138484993226,4cyclictest142969-21cron19:25:010
138484993226,3cyclictest204901-21cron22:00:010
138484993225,4cyclictest0-21swapper/023:30:010
138484993224,3cyclictest0-21swapper/000:05:480
138484993220,8cyclictest12650irq/32-eth021:50:450
138486993124,4cyclictest161510-21fschecks_time20:25:372
138486993124,3cyclictest0-21swapper/220:50:002
138485993124,4cyclictest264155-21unixbench_singl23:36:011
138485993124,3cyclictest0-21swapper/121:45:521
138485993122,5cyclictest150489-21systemd-run19:50:011
138484993122,6cyclictest254486-21ntpq23:20:460
138484993117,4cyclictest174342-21tail21:05:550
138484993113,11cyclictest12650irq/32-eth023:45:440
138486993021,6cyclictest421-21dbus-daemon22:06:252
138486993020,7cyclictest173118-21latency_hist21:05:032
138485993022,5cyclictest173490-21fschecks_count21:05:371
138485993022,4cyclictest0-21swapper/119:15:471
138485993021,6cyclictest156830-21aten_r2power_po20:10:341
138485993020,8cyclictest142382-21ntp_kernel_pll_19:20:441
138484993023,5cyclictest0-21swapper/023:00:580
138484993021,7cyclictest420-21avahi-daemon22:24:360
138484993018,8cyclictest12650irq/32-eth022:00:460
13848499301,14cyclictest263971-21timerandwakeup23:36:000
13848699297,13cyclictest1002-21mta-sts-daemon20:05:492
138486992912,13cyclictest421-21dbus-daemon19:40:012
138486992912,13cyclictest421-21dbus-daemon19:40:012
13848599295,14cyclictest428-21mta-sts-daemon20:35:441
138485992921,5cyclictest485-21in:imuxsock20:25:011
138484992924,3cyclictest151ktimers/020:55:500
138484992923,4cyclictest139914-21unixbench_singl19:10:550
138484992921,5cyclictest169956-21cron20:55:010
138484992920,7cyclictest1002-21mta-sts-daemon21:05:020
138484992919,7cyclictest236929-21unixbench_singl22:50:570
138485992821,2cyclictest0-21swapper/120:30:391
138484992823,2cyclictest0-21swapper/020:40:010
138484992821,4cyclictest296850-21needreboot00:30:430
138484992821,4cyclictest1001-21mta-sts-daemon19:45:450
138484992820,6cyclictest157277-21needreboot20:10:410
138484992816,9cyclictest0-21swapper/020:45:360
138486992718,6cyclictest143343-21aten_r2power_po19:25:362
138485992720,4cyclictest148002-21fschecks_count19:40:371
138485992715,9cyclictest169303-21ntp_kernel_err20:50:431
138485992619,5cyclictest139934-21sshd19:11:351
138484992621,2cyclictest0-21swapper/019:25:420
138484992619,4cyclictest163987-21systemd-run20:35:010
138486992518,4cyclictest151963-21munin-run19:55:012
138486992518,4cyclictest142972-21cron19:25:012
13848599254,12cyclictest428-21mta-sts-daemon20:45:441
138485992518,4cyclictest1003-21mta-sts-daemon20:05:031
138484992418,3cyclictest159331-21unixbench_multi20:15:560
138485992216,3cyclictest0-21swapper/119:10:011
138484992217,2cyclictest0-21swapper/019:35:020
138484992217,2cyclictest0-21swapper/019:35:020
138484992111,7cyclictest138622-21latency_hist19:10:020
13848799203,8cyclictest0-21swapper/300:35:463
13848799202,8cyclictest0-21swapper/322:30:453
13848799193,7cyclictest0-21swapper/322:45:503
13848799185,6cyclictest0-21swapper/323:55:473
13848799184,6cyclictest0-21swapper/323:35:543
13848799182,9cyclictest0-21swapper/323:10:483
13848799182,9cyclictest0-21swapper/323:10:483
13848799182,7cyclictest0-21swapper/319:10:423
13848799173,8cyclictest0-21swapper/322:15:473
13848799173,6cyclictest0-21swapper/322:25:453
13848799172,9cyclictest0-21swapper/322:35:483
13848799172,8cyclictest0-21swapper/300:25:473
13848799172,8cyclictest0-21swapper/300:05:413
13848799172,7cyclictest0-21swapper/322:55:483
13848799163,7cyclictest0-21swapper/322:00:523
13848799163,7cyclictest0-21swapper/321:50:483
13848799163,7cyclictest0-21swapper/321:40:473
13848799163,7cyclictest0-21swapper/300:10:463
13848799163,6cyclictest0-21swapper/321:25:453
13848799162,8cyclictest0-21swapper/323:50:473
13848799162,8cyclictest0-21swapper/323:45:443
13848799162,7cyclictest0-21swapper/322:40:453
13848799161,8cyclictest0-21swapper/321:45:423
13848799161,7cyclictest0-21swapper/323:15:433
13848799161,7cyclictest0-21swapper/320:55:463
13848799161,6cyclictest0-21swapper/320:05:453
13848799161,6cyclictest0-21swapper/319:30:463
13848799161,6cyclictest0-21swapper/319:30:463
13848799154,4cyclictest0-21swapper/319:35:483
13848799154,4cyclictest0-21swapper/319:35:483
13848799153,7cyclictest0-21swapper/323:00:503
13848799153,6cyclictest0-21swapper/323:40:423
13848799152,8cyclictest0-21swapper/323:20:433
13848799152,8cyclictest0-21swapper/323:05:463
13848799152,8cyclictest0-21swapper/323:05:463
13848799152,8cyclictest0-21swapper/321:30:483
13848799152,7cyclictest0-21swapper/320:15:463
13848799152,6cyclictest0-21swapper/319:55:423
13848799152,6cyclictest0-21swapper/319:15:433
13848799151,9cyclictest0-21swapper/300:00:433
13848799151,8cyclictest0-21swapper/321:15:423
13848799151,8cyclictest0-21swapper/321:15:423
13848799151,8cyclictest0-21swapper/321:10:503
13848799151,8cyclictest0-21swapper/300:15:443
13848799151,7cyclictest0-21swapper/321:20:433
13848799151,7cyclictest0-21swapper/320:50:433
13848799151,6cyclictest0-21swapper/321:05:433
13848799146,5cyclictest0-21swapper/321:55:473
13848799143,7cyclictest0-21swapper/300:20:533
13848799143,5cyclictest0-21swapper/320:20:453
13848799142,7cyclictest0-21swapper/322:20:363
13848799142,7cyclictest0-21swapper/322:05:453
13848799142,6cyclictest0-21swapper/323:25:433
13848799141,8cyclictest0-21swapper/322:50:363
13848799141,7cyclictest0-21swapper/323:30:423
13848799141,6cyclictest0-21swapper/319:20:443
13848799135,5cyclictest0-21swapper/319:40:433
13848799133,6cyclictest0-21swapper/300:30:403
13848799133,4cyclictest0-21swapper/320:00:433
13848799132,6cyclictest0-21swapper/322:10:353
13848799132,6cyclictest0-21swapper/322:10:353
13848799132,6cyclictest0-21swapper/321:35:463
13848799132,6cyclictest0-21swapper/319:50:443
13848799132,5cyclictest0-21swapper/319:45:443
13848799131,7cyclictest0-21swapper/320:25:443
13848799131,6cyclictest0-21swapper/321:00:433
13848799131,6cyclictest0-21swapper/320:35:443
13848799125,4cyclictest0-21swapper/320:30:483
13848799121,6cyclictest0-21swapper/320:10:453
13848799121,6cyclictest0-21swapper/319:25:453
13848799112,5cyclictest0-21swapper/320:45:423
13848799111,6cyclictest0-21swapper/320:40:363
13848799111,6cyclictest0-21swapper/320:40:363
1384879981,4cyclictest0-21swapper/319:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional