You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-12 - 06:31

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Wed Nov 12, 2025 00:39:53)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3276310995446,5cyclictest3374253-21processes22:34:130
3276310995446,5cyclictest3374253-21processes22:34:130
3276311995142,5cyclictest422-21dbus-daemon22:05:021
3276311995142,5cyclictest0-21swapper/121:24:111
3276311995043,4cyclictest3378752-21basename22:40:011
3276312994939,7cyclictest421-21avahi-daemon21:59:082
3276312994635,8cyclictest3408070-21sendmail23:20:012
3276311994638,5cyclictest3455745-21ssh00:25:011
3276310994635,8cyclictest1-21systemd00:00:030
3276312994424,10cyclictest955-21mta-sts-daemon21:14:082
3276312994424,10cyclictest955-21mta-sts-daemon21:14:082
3276310994437,4cyclictest3351749-21needreboot22:04:040
3276310994437,4cyclictest3301809-21systemd-run20:35:010
3276312994337,3cyclictest0-21swapper/222:05:012
3276312994337,3cyclictest0-21swapper/221:05:012
3276312994337,3cyclictest0-21swapper/200:30:022
3276312994336,3cyclictest0-21swapper/222:40:012
3276311994335,5cyclictest3419252-21cut23:35:011
3276310994334,5cyclictest3370938-21unixbench_multi22:29:180
3276310994334,5cyclictest3370938-21unixbench_multi22:29:180
3276312994235,4cyclictest3292785-21kworker/u8:4-nfsiod21:25:012
3276312994234,5cyclictest3434260-21latency_hist23:55:022
3276312994233,5cyclictest3389762-21idleruntime-cro22:55:012
3276312994232,7cyclictest3417631-21aten_r2power_fr23:33:562
3276312994232,6cyclictest0-21swapper/220:00:012
3276311994227,8cyclictest3312333-21diskmemload21:44:161
3276310994228,11cyclictest3452385-21awk00:20:010
3276312994134,3cyclictest0-21swapper/221:39:092
3276312994131,6cyclictest3437940-21systemd-run00:00:022
3276312994130,7cyclictest421-21avahi-daemon23:41:292
3276311994133,5cyclictest3465089-21ntpq00:39:131
3276311994133,5cyclictest3465089-21ntpq00:39:131
3276311994132,6cyclictest491-21gdbus21:49:231
3276310994133,5cyclictest3465091-21ssh00:39:130
3276310994133,5cyclictest3465091-21ssh00:39:130
3276310994129,10cyclictest3356165-21unixbench_multi22:09:220
3276311994032,4cyclictest3345462-21ssh21:55:001
3276312993932,4cyclictest0-21swapper/222:10:012
3276312993931,5cyclictest3319964-21cron21:20:012
3276312993930,6cyclictest3329451-21aten_r2power_po21:33:572
3276311993934,2cyclictest0-21swapper/100:00:021
3276311993931,4cyclictest0-21swapper/123:49:151
327631199392,33cyclictest3362967-21ssh22:19:091
327631199392,33cyclictest3362967-21ssh22:19:091
3276310993931,5cyclictest3430569-21latency_hist23:50:030
3276310993928,7cyclictest266-21systemd-journal21:21:470
3276312993829,6cyclictest3327045-21kworker/u8:6+xprtiod21:55:012
3276312993828,7cyclictest698-21cron00:40:012
3276312993828,7cyclictest698-21cron00:40:012
3276312993828,7cyclictest3452369-21munin-plugin-st00:20:012
327631199382,31cyclictest3355540-21ssh22:09:091
3276310993834,2cyclictest0-21swapper/023:09:230
3276310993834,2cyclictest0-21swapper/023:09:230
3276310993832,3cyclictest3393473-21systemctl23:00:010
3276312993730,4cyclictest0-21swapper/200:35:012
3276312993729,5cyclictest3415526-21cut23:30:012
3276312993729,5cyclictest3373633-21fschecks_count22:34:002
3276312993729,5cyclictest3373633-21fschecks_count22:34:002
3276312993729,3cyclictest0-21swapper/221:44:042
3276311993728,6cyclictest3287779-21ntpq19:49:131
3276311993728,6cyclictest3287779-21ntpq19:49:131
327631199372,30cyclictest3359365-21ssh22:14:121
3276310993731,4cyclictest3334900-21latency_hist21:40:030
3276310993729,5cyclictest3422482-21sh23:39:200
3276310993727,7cyclictest3382417-21cron22:45:010
3276312993628,5cyclictest3386139-21latency_hist22:50:022
3276312993628,5cyclictest1435531-21cupsd19:40:012
3276312993627,6cyclictest3276317-21systemd-run19:10:022
3276312993627,6cyclictest266-21systemd-journal23:15:012
3276312993626,7cyclictest421-21avahi-daemon23:38:222
327631299362,28cyclictest3447954-21ssh00:14:072
3276311993629,4cyclictest0-21swapper/100:19:071
3276311993629,3cyclictest0-21swapper/123:04:071
3276311993628,5cyclictest3414309-21latency23:29:031
3276311993626,7cyclictest3422957-21cron23:40:011
327631199362,24cyclictest3410761-21ssh23:24:091
3276310993630,4cyclictest0-21swapper/022:55:010
3276311993528,3cyclictest0-21swapper/122:24:081
3276311993528,3cyclictest0-21swapper/122:24:081
3276311993527,5cyclictest3391786-21aten_r2power_cu22:58:551
3276311993527,5cyclictest3319991-21sendmail21:20:011
3276311993527,5cyclictest3315916-21sed21:14:181
3276311993527,5cyclictest3315916-21sed21:14:181
3276311993527,5cyclictest3304785-21cron20:45:011
3276311993526,5cyclictest3292987-21latency_hist20:05:031
3276311993523,10cyclictest3370115-21latency22:29:021
3276311993523,10cyclictest3370115-21latency22:29:021
327631199352,24cyclictest3399776-21ssh23:09:131
327631199352,24cyclictest3399776-21ssh23:09:131
327631199351,29cyclictest3407204-21ssh23:19:121
3276310993529,4cyclictest3279199-21unixbench_multi19:19:180
3276310993528,5cyclictest3316310-21munin-plugin-st21:15:000
3276310993528,5cyclictest3316310-21munin-plugin-st21:15:000
3276310993525,7cyclictest3363393-21timerwakeupswit22:19:180
3276310993525,7cyclictest3363393-21timerwakeupswit22:19:180
3276310993525,7cyclictest3305680-21ntp_kernel_pll_20:49:070
327631099352,29cyclictest3448079-21ssh00:14:110
327631299343,24cyclictest3392600-21ssh22:59:122
3276312993428,3cyclictest0-21swapper/221:49:252
3276312993428,3cyclictest0-21swapper/221:29:212
3276312993428,3cyclictest0-21swapper/200:10:012
3276312993426,5cyclictest3295779-21cron20:15:012
3276312993424,6cyclictest3455519-21unixbench_singl00:24:212
3276311993428,3cyclictest0-21swapper/121:05:001
3276311993427,5cyclictest3375026-21/usr/sbin/munin22:35:011
3276311993427,5cyclictest3375026-21/usr/sbin/munin22:35:011
3276311993427,4cyclictest0-21swapper/120:34:071
3276311993426,5cyclictest3333832-21ntpq21:39:151
3276311993424,7cyclictest3459024-21sh00:30:021
3276310993428,3cyclictest0-21swapper/021:45:010
3276310993426,3cyclictest0-21swapper/022:39:120
3276310993423,8cyclictest3415153-21grep23:29:200
3276310993422,3cyclictest0-21swapper/023:04:130
327631099341,26cyclictest151ktimers/022:24:090
327631099341,26cyclictest151ktimers/022:24:090
3276312993323,6cyclictest3363965-21systemd-run22:20:022
3276312993323,6cyclictest3363965-21systemd-run22:20:022
3276312993323,6cyclictest3286821-21systemd-run19:45:012
3276311993327,3cyclictest0-21swapper/122:00:011
3276311993327,3cyclictest0-21swapper/100:10:011
3276311993326,3cyclictest0-21swapper/123:15:011
3276311993325,5cyclictest3309273-21munin-plugin-st21:00:001
3276311993325,5cyclictest1435558-21cups-browsed23:42:461
3276311993324,6cyclictest3382441-21systemd-run22:45:011
3276311993324,4cyclictest0-21swapper/100:00:191
3276311993323,7cyclictest3331015-21sh21:35:011
3276311993323,6cyclictest266-21systemd-journal19:55:011
3276310993327,4cyclictest3297252-21unixbench_singl20:19:170
3276310993326,5cyclictest421-21avahi-daemon00:01:250
3276310993326,5cyclictest3312356-21ssh21:10:020
3276310993326,4cyclictest956-21mta-sts-daemon22:14:310
3276310993326,4cyclictest422-21dbus-daemon23:55:020
3276310993326,4cyclictest3285279-21cron19:40:010
3276310993325,5cyclictest3327026-21head21:29:220
3276310993314,15cyclictest3454999-21ssh00:24:130
3276312993225,5cyclictest3276309-21cyclictest23:24:032
3276312993225,4cyclictest959-21mta-sts-daemon22:10:482
3276312993224,4cyclictest0-21swapper/220:44:142
3276312993223,6cyclictest266-21systemd-journal20:35:022
3276312993218,10cyclictest0-21swapper/200:04:062
327631299321,24cyclictest3396143-21ssh23:04:082
3276311993225,4cyclictest956-21mta-sts-daemon20:49:051
3276311993223,7cyclictest698-21cron19:40:011
3276311993223,6cyclictest957-21mta-sts-daemon22:46:321
3276311993223,5cyclictest0-21swapper/100:34:141
3276310993224,6cyclictest3457959-21/usr/sbin/munin00:29:060
3276310993224,5cyclictest3384483-21aten_r2power_en22:48:580
3276310993223,7cyclictest3403116-21kernelversion23:14:040
3276310993222,8cyclictest3438036-21cups-browsed00:32:180
3276310993216,4cyclictest0-21swapper/023:15:270
3276312993124,4cyclictest3288948-21kernelversion19:54:012
3276312993123,5cyclictest474-21in:imuxsock20:50:012
3276312993122,7cyclictest3283054-21missed_timers19:34:022
327631299312,21cyclictest3381397-21ssh22:44:062
3276312993113,13cyclictest1435531-21cupsd20:19:062
3276311993125,3cyclictest0-21swapper/122:54:071
3276311993124,4cyclictest3327382-21systemd-run21:30:021
3276311993123,5cyclictest3286687-21unixbench_multi19:44:201
3276311993121,7cyclictest3290608-21needreboot19:59:041
3276310993124,5cyclictest3287343-21fschecks_time19:49:020
3276310993124,5cyclictest3287343-21fschecks_time19:49:020
3276310993123,5cyclictest422-21dbus-daemon21:33:580
3276312993023,3cyclictest0-21swapper/223:08:572
3276312993023,3cyclictest0-21swapper/223:08:572
3276312993020,6cyclictest3428046-21sshd23:47:222
3276311993022,5cyclictest3303282-21munin-run20:40:011
327631099307,17cyclictest12450irq/32-eth021:49:170
3276310993024,4cyclictest3411769-21grep23:25:010
3276310993024,4cyclictest3298871-21latency_hist20:25:020
3276310993023,4cyclictest3344776-21mailstats21:54:220
3276310993023,4cyclictest3292790-21cron20:05:010
3276310993022,5cyclictest0-21swapper/023:35:020
3276310993015,11cyclictest3319146-21ssh21:19:130
3276312992921,6cyclictest3312247-21grep21:09:222
3276312992920,6cyclictest453-21systemd-logind22:30:012
3276312992920,6cyclictest453-21systemd-logind22:30:012
3276312992917,9cyclictest3306814-21cut20:54:032
3276312992917,9cyclictest3306814-21cut20:54:032
3276311992924,2cyclictest0-21swapper/119:20:011
3276311992923,4cyclictest959-21mta-sts-daemon20:50:541
3276311992923,4cyclictest959-21mta-sts-daemon20:50:541
3276311992923,3cyclictest0-21swapper/119:10:021
3276311992920,6cyclictest3434172-21cron23:55:011
3276310992923,3cyclictest0-21swapper/000:09:050
327631099292,22cyclictest3425760-21ssh23:44:070
3276310992921,5cyclictest3279856-21fschecks_time19:23:590
3276310992921,3cyclictest957-21mta-sts-daemon20:29:000
327631099291,24cyclictest657-21snmpd20:44:120
327631299283,15cyclictest3366620-21sh22:24:062
327631299283,15cyclictest3366620-21sh22:24:062
3276312992820,5cyclictest3297837-21fschecks_count20:24:002
3276311992823,2cyclictest0-21swapper/120:15:011
3276311992821,5cyclictest0-21swapper/121:10:011
3276311992821,4cyclictest0-21swapper/100:14:131
3276310992823,3cyclictest959-21mta-sts-daemon21:57:300
3276310992819,6cyclictest3302928-21/usr/sbin/munin20:39:180
3276312992721,3cyclictest0-21swapper/220:40:022
3276312992719,5cyclictest3291547-21/usr/sbin/munin20:04:102
3276312992719,5cyclictest3279278-21vmstat19:19:192
3276311992719,4cyclictest956-21mta-sts-daemon20:29:071
3276311992718,7cyclictest3297020-21timerandwakeup20:19:161
3276311992718,6cyclictest1435558-21cups-browsed19:24:031
3276310992721,4cyclictest3294247-21unixbench_singl20:09:190
3276310992720,4cyclictest3286791-21cron19:45:010
3276310992719,2cyclictest0-21swapper/020:54:210
3276310992719,2cyclictest0-21swapper/020:54:210
3276310992718,7cyclictest955-21mta-sts-daemon19:14:080
3276312992621,2cyclictest0-21swapper/219:25:012
327631299261,19cyclictest3309281-21awk21:00:002
3276311992618,5cyclictest1435558-21cups-browsed20:22:451
3276310992620,4cyclictest3288829-21grep19:53:580
3276312992519,3cyclictest0-21swapper/219:15:022
3276312992517,4cyclictest3293359-21gltestperf20:09:012
3276312992514,7cyclictest266-21systemd-journal20:30:012
3276311992518,4cyclictest3282293-21cron19:30:011
3276312992417,5cyclictest955-21mta-sts-daemon19:49:022
3276312992417,5cyclictest955-21mta-sts-daemon19:49:022
3276310992412,9cyclictest0-21swapper/019:29:110
327631299232,15cyclictest3281787-21ntpq19:29:102
3276311992317,4cyclictest956-21mta-sts-daemon20:07:221
3276311992315,5cyclictest3277204-21ntpq19:14:071
3276311992311,9cyclictest3283126-21needreboot19:34:031
327631099231,8cyclictest3282957-21kernelversion19:34:010
3276310992317,3cyclictest0-21swapper/019:10:030
3276310992316,5cyclictest1435558-21cups-browsed20:14:090
327631099228,11cyclictest3310381-21sendmail_mailst21:04:140
3276310992218,2cyclictest0-21swapper/020:00:010
3276310992217,3cyclictest151ktimers/020:59:100
327631399172,8cyclictest0-21swapper/321:59:173
327631399161,8cyclictest0-21swapper/321:14:093
327631399161,8cyclictest0-21swapper/321:14:093
327631399161,7cyclictest0-21swapper/322:04:093
327631399151,8cyclictest0-21swapper/300:24:023
327631399151,7cyclictest0-21swapper/321:54:103
327631399151,7cyclictest0-21swapper/321:09:473
327631399151,4cyclictest0-21swapper/320:55:023
327631399151,4cyclictest0-21swapper/320:55:023
327631399151,3cyclictest0-21swapper/322:24:093
327631399151,3cyclictest0-21swapper/322:24:093
327631399143,5cyclictest0-21swapper/321:49:153
327631399143,5cyclictest0-21swapper/300:29:133
327631399142,6cyclictest0-21swapper/321:24:143
327631399142,5cyclictest0-21swapper/321:19:063
327631399141,7cyclictest0-21swapper/323:19:063
327631399141,7cyclictest0-21swapper/323:14:083
327631399141,7cyclictest0-21swapper/322:14:083
327631399141,7cyclictest0-21swapper/320:59:093
327631399141,7cyclictest0-21swapper/319:54:063
327631399141,6cyclictest0-21swapper/323:34:063
327631399141,6cyclictest0-21swapper/322:19:133
327631399141,6cyclictest0-21swapper/322:19:133
327631399141,6cyclictest0-21swapper/320:29:053
327631399141,6cyclictest0-21swapper/319:29:063
327631399141,6cyclictest0-21swapper/300:09:073
327631399141,5cyclictest0-21swapper/320:39:123
327631399133,5cyclictest0-21swapper/323:44:103
327631399133,4cyclictest0-21swapper/320:19:063
327631399132,4cyclictest0-21swapper/320:04:113
327631399131,7cyclictest0-21swapper/323:49:133
327631399131,7cyclictest0-21swapper/322:49:113
327631399131,7cyclictest0-21swapper/321:29:123
327631399131,6cyclictest0-21swapper/323:59:053
327631399131,6cyclictest0-21swapper/323:09:103
327631399131,6cyclictest0-21swapper/323:09:103
327631399131,6cyclictest0-21swapper/321:33:573
327631399131,6cyclictest0-21swapper/320:24:133
327631399131,6cyclictest0-21swapper/320:09:143
327631399131,6cyclictest0-21swapper/319:59:073
327631399131,6cyclictest0-21swapper/300:39:073
327631399131,6cyclictest0-21swapper/300:39:073
327631399131,6cyclictest0-21swapper/300:14:103
327631399131,5cyclictest0-21swapper/319:44:073
327631399130,7cyclictest0-21swapper/300:34:133
327631399123,5cyclictest0-21swapper/323:29:123
327631399122,6cyclictest0-21swapper/322:43:563
327631399122,6cyclictest0-21swapper/322:39:113
327631399122,6cyclictest0-21swapper/300:05:013
327631399122,5cyclictest0-21swapper/300:19:023
327631399121,7cyclictest0-21swapper/323:24:093
327631399121,7cyclictest0-21swapper/321:39:043
327631399121,6cyclictest0-21swapper/323:39:143
327631399121,6cyclictest0-21swapper/322:59:073
327631399121,6cyclictest0-21swapper/322:54:083
327631399121,6cyclictest0-21swapper/321:04:103
327631399121,6cyclictest0-21swapper/320:14:093
327631399121,6cyclictest0-21swapper/319:19:053
327631399121,5cyclictest0-21swapper/319:49:103
327631399121,5cyclictest0-21swapper/319:49:103
327631399121,5cyclictest0-21swapper/319:24:053
327631399113,5cyclictest0-21swapper/322:34:093
327631399113,5cyclictest0-21swapper/322:34:093
327631399112,5cyclictest0-21swapper/319:40:013
327631399112,4cyclictest0-21swapper/322:29:093
327631399112,4cyclictest0-21swapper/322:29:093
327631399112,4cyclictest0-21swapper/320:34:053
327631399111,7cyclictest0-21swapper/323:54:073
327631399111,6cyclictest0-21swapper/323:03:583
327631399111,5cyclictest0-21swapper/321:44:023
327631399111,5cyclictest0-21swapper/320:49:123
327631399101,6cyclictest0-21swapper/320:44:113
327631399101,5cyclictest0-21swapper/322:08:593
327631399100,5cyclictest0-21swapper/319:14:063
32763139982,4cyclictest0-21swapper/319:34:043
32763139981,4cyclictest0-21swapper/319:10:033
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional