You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-28 - 02:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Tue Jul 28, 2026 00:43:10)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
704683996048,8cyclictest816811-21latency_hist23:15:031
704684995139,7cyclictest1002-21mta-sts-daemon19:10:482
70468399513,38cyclictest781399-21ssh22:15:471
70468399494,9cyclictest799646-21ssh22:45:481
704683994922,5cyclictest834945-21latency_hist23:45:021
704683994922,5cyclictest834945-21latency_hist23:45:021
704682994942,4cyclictest792463-21cut22:35:010
704684994734,9cyclictest846858-21/usr/sbin/munin00:05:022
704684994734,9cyclictest846858-21/usr/sbin/munin00:05:022
704683994638,5cyclictest265-21systemd-journal20:15:011
704683994520,3cyclictest0-21swapper/100:35:501
704683994519,5cyclictest3196310-21cups-browsed21:57:581
704684994435,5cyclictest713662-21munin-plugin-st19:40:012
704684994435,5cyclictest713662-21munin-plugin-st19:40:012
704683994424,4cyclictest784936-21timerwakeupswit22:20:581
704682994436,6cyclictest0-21swapper/000:00:020
704682994435,3cyclictest0-21swapper/000:25:530
704684994333,6cyclictest733245-21systemd-run20:45:012
704684994333,6cyclictest707667-21grep19:20:012
704683994333,6cyclictest859040-21latency_hist00:25:021
70468399433,16cyclictest665-21snmpd19:50:491
704682994336,4cyclictest768195-21kworker/u8:10-rpciod22:05:410
704682994329,11cyclictest808384-21fschecks_count23:00:440
704682994317,18cyclictest12650irq/32-eth000:15:480
704682994313,20cyclictest805787-21sh22:55:520
704684994235,4cyclictest721172-21idleruntime-cro20:05:012
704684994235,4cyclictest721172-21idleruntime-cro20:05:012
704684994233,5cyclictest825801-21systemd-run23:30:022
704683994234,5cyclictest0-21swapper/100:30:011
704683994218,3cyclictest0-21swapper/122:10:481
704682994237,2cyclictest0-21swapper/021:25:500
704682994235,4cyclictest765121-21cron21:50:010
704682994235,4cyclictest736208-21ls20:55:010
704682994234,5cyclictest420-21avahi-daemon00:37:150
704682994233,7cyclictest742173-21switchtime21:10:580
70468299421,34cyclictest766293-21ssh21:50:500
704684994133,5cyclictest855864-21awk00:20:012
704684994127,11cyclictest830129-21mailstats23:35:562
704683994134,4cyclictest1002-21mta-sts-daemon22:50:301
704683994132,6cyclictest750448-21aten_r2power_po21:25:391
704683994122,3cyclictest0-21swapper/100:35:031
704682994133,6cyclictest839980-21kworker/u8:14-xprtiod00:10:010
704682994133,6cyclictest839980-21kworker/u8:14-xprtiod00:10:010
704682994124,12cyclictest443-21gpm21:15:470
704682994124,12cyclictest443-21gpm21:15:470
70468499405,25cyclictest784445-21ssh22:20:452
704683994032,5cyclictest733245-21systemd-run20:45:011
704683994032,4cyclictest0-21swapper/121:10:591
704684993931,5cyclictest852885-21systemd00:15:012
704683993931,5cyclictest265-21systemd-journal23:10:011
704683993931,4cyclictest0-21swapper/121:25:001
704683993931,4cyclictest0-21swapper/121:25:001
704683993931,3cyclictest0-21swapper/123:40:001
704683993930,6cyclictest810694-21systemd-run23:05:011
704682993923,12cyclictest151ktimers/019:25:460
704682993923,12cyclictest151ktimers/019:25:460
70468499383,26cyclictest802798-21ssh22:50:522
704684993831,3cyclictest0-21swapper/223:55:502
704684993830,5cyclictest815385-21grep23:11:002
704684993829,6cyclictest0-21swapper/222:35:022
704684993827,6cyclictest867920-21systemd-run00:40:012
704684993824,10cyclictest847180-21sshd00:05:222
704684993824,10cyclictest847180-21sshd00:05:222
704683993832,3cyclictest0-21swapper/121:35:541
704683993832,3cyclictest0-21swapper/121:35:541
704683993830,5cyclictest421-21dbus-daemon23:50:011
704683993825,9cyclictest853874-21needreboot00:15:431
704682993832,3cyclictest0-21swapper/019:10:510
704682993830,5cyclictest786968-21aten_r2power_po22:25:390
704682993830,5cyclictest786968-21aten_r2power_po22:25:390
704682993827,7cyclictest721324-21latency_hist20:05:040
704682993827,7cyclictest721324-21latency_hist20:05:040
704682993825,4cyclictest733215-21cron20:45:010
704684993729,5cyclictest724157-21cron20:15:012
704684993727,7cyclictest1000-21mta-sts-daemon22:59:222
704684993727,6cyclictest567-21gdbus22:40:022
704683993728,6cyclictest845529-21unixbench_multi00:00:591
704683993728,6cyclictest845529-21unixbench_multi00:00:591
704682993732,3cyclictest0-21swapper/022:40:020
704682993731,4cyclictest846873-21systemd-run00:05:020
704682993731,4cyclictest846873-21systemd-run00:05:020
704682993728,7cyclictest822785-21sh23:25:010
704682993726,8cyclictest759901-21kernelversion21:40:420
704682993721,7cyclictest12650irq/32-eth023:40:480
704682993721,7cyclictest12650irq/32-eth023:40:480
704684993628,5cyclictest832508-21cut23:40:412
704684993628,5cyclictest832508-21cut23:40:412
704684993628,5cyclictest718177-21true19:55:012
704684993628,4cyclictest1001-21mta-sts-daemon21:26:062
704684993627,4cyclictest0-21swapper/222:15:572
704684993626,3cyclictest0-21swapper/223:50:502
704683993629,3cyclictest0-21swapper/122:00:561
704683993628,5cyclictest740713-21diskmemload21:55:011
704683993628,5cyclictest1003-21mta-sts-daemon22:28:151
704683993628,5cyclictest1003-21mta-sts-daemon22:28:151
704682993631,3cyclictest740827-21latency_hist21:10:020
704682993629,4cyclictest1002-21mta-sts-daemon22:45:030
704682993629,4cyclictest1001-21mta-sts-daemon23:36:230
704682993628,5cyclictest817785-21needreboot23:15:450
704682993627,7cyclictest753264-21sshd21:30:220
704682993627,7cyclictest1003-21mta-sts-daemon22:13:590
704682993627,6cyclictest835355-21aten_r2power_po23:45:380
704682993624,9cyclictest0-21swapper/022:05:010
704684993527,3cyclictest0-21swapper/223:25:022
704684993526,6cyclictest856792-21latency00:20:442
704684993524,7cyclictest3196310-21cups-browsed21:48:472
704683993529,3cyclictest0-21swapper/123:55:011
704683993528,4cyclictest746820-21ssh21:20:011
704683993528,4cyclictest746820-21ssh21:20:011
70468399352,27cyclictest821072-21ssh23:20:551
704683993516,10cyclictest763201-21sh21:45:451
70468299356,20cyclictest12650irq/32-eth019:35:500
70468299356,20cyclictest12650irq/32-eth019:35:500
70468299355,19cyclictest428-21mta-sts-daemon20:10:500
704682993529,4cyclictest824219-21switchtime23:26:000
704682993526,6cyclictest1002-21mta-sts-daemon22:19:260
704682993524,8cyclictest0-21swapper/021:56:040
704682993521,11cyclictest0-21swapper/019:20:010
704684993427,3cyclictest0-21swapper/222:00:002
704684993426,5cyclictest819785-21idleruntime-cro23:20:012
704684993426,3cyclictest0-21swapper/222:00:552
704684993425,6cyclictest421-21dbus-daemon19:35:012
70468499342,24cyclictest787614-21ssh22:25:532
70468499342,24cyclictest787614-21ssh22:25:532
704683993430,2cyclictest0-21swapper/121:35:011
704683993427,4cyclictest828780-21munin-plugin-st23:35:001
704683993427,3cyclictest0-21swapper/122:35:381
70468299344,24cyclictest857163-21grep00:20:550
704682993425,4cyclictest0-21swapper/022:50:490
704682993413,14cyclictest12650irq/32-eth022:20:450
704684993327,3cyclictest0-21swapper/223:10:012
704684993326,4cyclictest1001-21mta-sts-daemon22:14:332
704684993323,7cyclictest420-21avahi-daemon00:32:402
70468499332,24cyclictest860106-21ssh00:25:542
70468499331,27cyclictest808891-21ssh23:00:582
704683993326,5cyclictest805331-21fschecks_count22:55:421
704683993325,5cyclictest420-21avahi-daemon22:40:461
70468399332,24cyclictest823876-21ssh23:25:491
704682993328,3cyclictest737792-21latency_hist21:00:020
704682993328,3cyclictest1003-21mta-sts-daemon00:34:120
704682993327,3cyclictest0-21swapper/023:50:590
704682993322,8cyclictest756961-21missed_timers21:35:440
704682993322,8cyclictest756961-21missed_timers21:35:440
704684993227,2cyclictest0-21swapper/223:50:002
704684993226,3cyclictest0-21swapper/221:35:382
704684993226,3cyclictest0-21swapper/221:35:382
704684993226,3cyclictest0-21swapper/221:15:012
704684993225,3cyclictest0-21swapper/221:55:012
704684993223,7cyclictest760086-21needreboot21:40:452
704684993223,6cyclictest1-21systemd20:30:022
70468499322,22cyclictest748085-21ssh21:20:532
70468499322,22cyclictest748085-21ssh21:20:532
70468399324,22cyclictest760263-21ssh21:40:521
70468399323,20cyclictest1002-21mta-sts-daemon19:10:481
704683993224,3cyclictest0-21swapper/122:05:521
704683993223,5cyclictest722682-21systemd-run20:10:011
704682993225,5cyclictest798877-21kworker/u8:13+xprtiod23:06:000
704682993224,3cyclictest0-21swapper/022:40:570
704682993222,6cyclictest420-21avahi-daemon21:23:460
704682993222,6cyclictest420-21avahi-daemon21:23:460
704684993126,2cyclictest0-21swapper/219:45:002
704684993124,4cyclictest1001-21mta-sts-daemon22:45:412
704684993118,9cyclictest774817-21basename22:05:382
704683993124,4cyclictest703424-21kworker/u8:15-rpciod19:10:011
704683993124,3cyclictest0-21swapper/100:06:001
704683993124,3cyclictest0-21swapper/100:06:001
704683993123,5cyclictest819773-21cron23:20:011
704682993125,4cyclictest734714-21idleruntime-cro20:50:010
704682993124,4cyclictest852855-21debian-sa100:15:000
70468499304,20cyclictest744848-21ssh21:15:472
70468499304,20cyclictest744848-21ssh21:15:472
704684993024,3cyclictest0-21swapper/220:10:002
704683993025,2cyclictest0-21swapper/120:55:421
704683993023,4cyclictest1001-21mta-sts-daemon19:16:161
704683993020,7cyclictest740029-21needreboot21:05:471
70468299303,10cyclictest815283-21timerwakeupswit23:10:590
704682993018,9cyclictest0-21swapper/019:40:440
704684992924,2cyclictest0-21swapper/220:20:012
704684992922,4cyclictest827501-21unixbench_singl23:31:012
704684992919,6cyclictest0-21swapper/222:40:572
704683992923,3cyclictest0-21swapper/119:45:011
704683992922,4cyclictest1003-21mta-sts-daemon20:20:471
704683992922,4cyclictest0-21swapper/100:00:021
70468399292,18cyclictest790567-21ssh22:30:501
704683992921,5cyclictest421-21dbus-daemon20:27:241
70468399291,24cyclictest850969-21sh00:10:481
704682992923,4cyclictest826435-21cpuspeed23:30:390
704682992923,4cyclictest704681-21cyclictest19:45:490
704684992820,5cyclictest754428-21timerandwakeup21:30:582
704683992823,2cyclictest0-21swapper/119:40:011
704683992823,2cyclictest0-21swapper/119:40:011
704683992822,3cyclictest721396-21kworker/u8:5-xprtiod20:40:011
704683992820,5cyclictest729510-21ntp_kernel_err20:30:451
704683992820,5cyclictest713703-21kworker/u8:10-xprtiod20:00:011
704682992823,3cyclictest709120-21unixbench_singl19:20:580
704682992823,3cyclictest709120-21unixbench_singl19:20:580
704684992722,2cyclictest0-21swapper/219:50:012
704684992719,5cyclictest727035-21unixbench-2d20:20:582
704684992719,5cyclictest709196-21systemd-run19:25:012
704684992719,5cyclictest709196-21systemd-run19:25:012
704683992719,5cyclictest3196310-21cups-browsed19:47:551
704683992718,6cyclictest734729-21systemd-run20:50:011
704683992718,6cyclictest708211-21uniq19:20:391
704683992718,6cyclictest708211-21uniq19:20:391
704682992721,4cyclictest724619-21cpuspeed20:15:390
704682992720,4cyclictest721695-21fschecks_count20:05:400
704682992719,5cyclictest485-21in:imuxsock19:35:000
704684992619,3cyclictest0-21swapper/220:56:002
704684992618,6cyclictest709575-21aten_r2power_vo19:25:382
704684992618,6cyclictest709575-21aten_r2power_vo19:25:382
704684992618,5cyclictest731658-21munin-plugin-st20:40:012
704684992618,5cyclictest3196310-21cups-browsed20:33:272
704684992616,7cyclictest738732-21proc_pri21:01:092
704683992620,3cyclictest0-21swapper/120:01:011
704683992620,3cyclictest0-21swapper/120:01:011
704683992618,5cyclictest709405-21/usr/sbin/munin19:25:571
704683992618,5cyclictest709405-21/usr/sbin/munin19:25:571
704682992621,2cyclictest0-21swapper/019:53:220
704682992620,4cyclictest730154-21cron20:35:010
704684992518,4cyclictest1000-21mta-sts-daemon20:52:232
70468399255,11cyclictest1000-21mta-sts-daemon20:50:461
704683992517,5cyclictest738241-21fschecks_count21:00:471
704682992518,4cyclictest703340-21kworker/u8:11-xprtiod19:10:010
704682992514,8cyclictest0-21swapper/021:00:470
704683992419,2cyclictest0-21swapper/120:15:381
70468299244,14cyclictest428-21mta-sts-daemon20:25:510
70468299242,12cyclictest151ktimers/020:20:470
70468599231,15cyclictest0-21swapper/321:50:533
70468499238,11cyclictest421-21dbus-daemon21:05:552
704683992317,3cyclictest696480-21kworker/u8:6-xprtiod19:30:471
70468599222,9cyclictest0-21swapper/323:00:513
70468499224,12cyclictest428-21mta-sts-daemon20:45:482
70468499221,16cyclictest719020-21ntpq19:55:482
70468299222,8cyclictest718427-21aten_r2power_cu19:55:380
70468299221,15cyclictest731013-21ntpq20:35:460
70468599212,10cyclictest0-21swapper/300:20:553
70468599211,14cyclictest0-21swapper/321:30:473
704684992011,6cyclictest704703-21latency_hist19:10:022
70468599183,8cyclictest0-21swapper/321:20:493
70468599183,8cyclictest0-21swapper/321:20:493
70468599182,9cyclictest0-21swapper/300:25:533
70468599182,11cyclictest0-21swapper/321:40:513
70468599182,10cyclictest0-21swapper/323:50:553
70468599182,10cyclictest0-21swapper/300:15:453
70468599181,9cyclictest0-21swapper/322:50:523
70468599181,9cyclictest0-21swapper/322:05:513
70468599181,11cyclictest0-21swapper/322:00:503
70468599181,10cyclictest0-21swapper/323:45:543
70468599181,10cyclictest0-21swapper/321:05:523
70468599172,8cyclictest0-21swapper/321:10:453
70468599171,9cyclictest0-21swapper/322:30:513
70468599171,9cyclictest0-21swapper/322:20:463
70468599171,9cyclictest0-21swapper/322:15:503
70468599171,11cyclictest0-21swapper/323:40:463
70468599171,11cyclictest0-21swapper/323:40:463
70468599171,10cyclictest0-21swapper/323:35:473
70468599171,10cyclictest0-21swapper/322:45:523
70468599171,10cyclictest0-21swapper/321:15:553
70468599171,10cyclictest0-21swapper/321:15:553
70468599163,8cyclictest0-21swapper/321:25:503
70468599162,9cyclictest0-21swapper/323:30:413
70468599162,9cyclictest0-21swapper/322:40:473
70468599162,9cyclictest0-21swapper/319:10:513
70468599162,9cyclictest0-21swapper/300:35:373
70468599162,8cyclictest0-21swapper/321:55:423
70468599162,8cyclictest0-21swapper/321:35:483
70468599162,8cyclictest0-21swapper/321:35:483
70468599162,7cyclictest0-21swapper/320:40:243
70468599162,7cyclictest0-21swapper/320:10:013
70468599162,7cyclictest0-21swapper/319:30:473
70468599162,6cyclictest0-21swapper/320:30:473
70468599161,9cyclictest0-21swapper/300:05:483
70468599161,9cyclictest0-21swapper/300:05:483
70468599161,8cyclictest0-21swapper/323:20:493
70468599161,10cyclictest0-21swapper/323:15:463
70468599152,7cyclictest0-21swapper/323:25:483
70468599152,7cyclictest0-21swapper/323:05:503
70468599151,9cyclictest0-21swapper/322:55:533
70468599151,9cyclictest0-21swapper/300:30:473
70468599151,8cyclictest0-21swapper/320:50:513
70468599151,8cyclictest0-21swapper/319:16:193
70468599151,8cyclictest0-21swapper/300:00:373
70468599151,8cyclictest0-21swapper/300:00:373
70468599151,7cyclictest0-21swapper/319:55:493
70468599151,10cyclictest0-21swapper/323:10:513
70468599143,7cyclictest0-21swapper/321:46:223
70468599143,6cyclictest0-21swapper/322:25:513
70468599143,6cyclictest0-21swapper/322:25:513
70468599142,8cyclictest0-21swapper/323:55:403
70468599142,7cyclictest0-21swapper/322:10:403
70468599141,8cyclictest0-21swapper/322:35:493
70468599141,8cyclictest0-21swapper/320:10:533
70468599141,7cyclictest0-21swapper/319:40:533
70468599132,8cyclictest0-21swapper/300:10:453
70468599132,6cyclictest0-21swapper/320:20:473
70468599131,7cyclictest0-21swapper/320:35:493
70468599131,7cyclictest0-21swapper/320:15:473
70468599131,7cyclictest0-21swapper/320:00:503
70468599131,7cyclictest0-21swapper/320:00:503
70468599131,7cyclictest0-21swapper/319:35:463
70468599131,7cyclictest0-21swapper/319:35:463
70468599131,6cyclictest0-21swapper/319:50:453
70468599131,6cyclictest0-21swapper/319:45:473
70468599121,7cyclictest0-21swapper/321:00:593
70468599121,7cyclictest0-21swapper/320:45:493
70468599121,6cyclictest0-21swapper/319:25:453
70468599121,6cyclictest0-21swapper/319:25:453
70468599121,6cyclictest0-21swapper/319:20:453
70468599121,6cyclictest0-21swapper/319:20:453
70468599114,4cyclictest0-21swapper/320:25:473
70468599111,6cyclictest0-21swapper/320:55:503
7046859991,4cyclictest0-21swapper/319:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional