You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-07 - 15:57

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Sun Jun 07, 2026 12:42:53)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3312772996150,7cyclictest420-21avahi-daemon11:41:060
3312772996150,7cyclictest420-21avahi-daemon11:41:060
3312772995950,6cyclictest439-21systemd-logind12:35:020
3312774995848,5cyclictest3445921-21kworker/u8:7-writeback11:45:062
3312774995848,5cyclictest3445921-21kworker/u8:7-writeback11:45:062
3312774995645,7cyclictest3379935-21awk10:00:012
331277499484,31cyclictest3471949-21ssh12:26:042
331277499473,35cyclictest3348761-21diskmemload10:06:042
3312772994740,3cyclictest0-21swapper/011:50:010
3312772994740,3cyclictest0-21swapper/011:50:010
3312772994736,8cyclictest3364361-21basename09:35:010
3312772994733,11cyclictest3462551-21missed_timers12:11:050
331277499462,37cyclictest3457345-21ssh12:02:002
331277399464,34cyclictest3371911-21ssh09:46:081
3312772994637,5cyclictest420-21avahi-daemon10:21:040
3312772994637,5cyclictest420-21avahi-daemon10:21:040
3312774994436,5cyclictest0-21swapper/208:40:012
331277399442,30cyclictest3459690-21ssh12:06:101
3312772994335,5cyclictest3417130-21getprofile11:00:000
3312774994234,5cyclictest3442765-21ssh11:40:002
3312774994234,5cyclictest3442765-21ssh11:40:002
3312774994232,6cyclictest3402265-21aten_r2power_vo10:35:562
3312774994232,6cyclictest3402265-21aten_r2power_vo10:35:562
3312773994233,6cyclictest3339718-21/usr/sbin/munin08:36:171
3312774994134,3cyclictest0-21swapper/208:30:012
3312774994133,5cyclictest3330764-21cut08:10:012
3312773994133,4cyclictest0-21swapper/112:26:081
3312773994129,9cyclictest3477860-21kernelversion12:36:001
3312773994124,14cyclictest271ktimers/111:11:041
3312772994134,4cyclictest3410925-21systemd-run10:50:010
3312772994134,4cyclictest3410925-21systemd-run10:50:010
3312774994027,5cyclictest0-21swapper/209:16:072
3312773994031,5cyclictest3390732-21cut10:16:171
3312773994031,5cyclictest3390732-21cut10:16:171
3312773994030,6cyclictest2524656-21cups-browsed10:08:321
3312772994033,5cyclictest3312832-21latency_hist07:10:010
3312772994032,5cyclictest3456633-21proc_pri12:01:140
3312772994026,10cyclictest3332269-21sh08:15:010
3312774993931,5cyclictest3467632-21cut12:20:012
3312774993931,5cyclictest0-21swapper/211:45:022
3312774993931,5cyclictest0-21swapper/211:45:022
331277499392,32cyclictest3387469-21ssh10:11:082
3312773993933,3cyclictest0-21swapper/112:01:031
3312773993931,4cyclictest0-21swapper/110:01:021
3312773993930,6cyclictest420-21avahi-daemon09:21:021
3312772993930,6cyclictest2524652-21cupsd11:15:580
3312774993829,6cyclictest3430378-21sendmail11:20:012
3312774993829,4cyclictest0-21swapper/209:35:022
331277499381,30cyclictest3412078-21ssh10:51:032
331277499381,30cyclictest3412078-21ssh10:51:032
3312773993832,3cyclictest0-21swapper/110:35:001
3312773993832,3cyclictest0-21swapper/110:35:001
3312773993832,3cyclictest0-21swapper/110:00:011
3312773993830,5cyclictest3444015-21ntp_kernel_pll_11:41:041
3312773993830,5cyclictest3444015-21ntp_kernel_pll_11:41:041
3312772993831,4cyclictest3373731-21systemd-run09:50:010
3312772993829,5cyclictest3437975-21proc_pri11:31:120
3312772993829,5cyclictest3437975-21proc_pri11:31:120
3312774993729,5cyclictest3476872-21cron12:35:012
3312774993729,3cyclictest0-21swapper/209:41:022
3312774993728,4cyclictest0-21swapper/209:30:012
3312774993727,7cyclictest2524656-21cups-browsed12:08:042
3312774993726,7cyclictest2524656-21cups-browsed09:22:292
3312773993732,2cyclictest0-21swapper/111:20:011
3312773993729,5cyclictest3335787-21fschecks_count08:25:561
3312773993728,6cyclictest3326243-21cron07:55:001
3312773993728,6cyclictest2524656-21cups-browsed11:36:091
3312773993728,6cyclictest2524656-21cups-browsed11:36:091
3312773993727,4cyclictest0-21swapper/110:41:071
3312773993711,20cyclictest3434656-21ssh11:26:021
331277299373,29cyclictest3359458-21ssh09:26:080
3312772993729,5cyclictest421-21dbus-daemon10:55:010
3312772993729,5cyclictest421-21dbus-daemon10:55:010
3312772993728,6cyclictest2524656-21cups-browsed08:16:160
3312772993727,7cyclictest3355627-21aten_r2power_fr09:20:530
3312774993629,3cyclictest0-21swapper/211:55:012
3312774993629,3cyclictest0-21swapper/211:55:012
3312774993628,5cyclictest3431233-21fschecks_time11:20:592
3312774993628,5cyclictest3431233-21fschecks_time11:20:592
3312774993628,5cyclictest0-21swapper/207:31:062
3312774993624,7cyclictest3415231-21sh10:56:052
3312773993629,3cyclictest0-21swapper/110:55:011
3312773993629,3cyclictest0-21swapper/110:55:011
3312773993627,6cyclictest261-21systemd-journal11:00:441
3312773993623,9cyclictest3467624-21idleruntime-cro12:20:011
3312772993630,4cyclictest3381723-21tail10:01:160
331277299362,31cyclictest3419302-21sh11:01:100
331277299361,15cyclictest3350982-21kworker/u8:3+rpciod09:41:020
3312774993527,5cyclictest3406547-21unixbench_singl10:41:172
3312774993527,5cyclictest3348772-21ssh09:10:012
3312774993527,5cyclictest1055-21mta-sts-daemon11:25:462
3312774993527,3cyclictest0-21swapper/211:11:122
3312774993526,6cyclictest3409542-21unixbench-2d10:46:172
3312774993526,6cyclictest3409542-21unixbench-2d10:46:172
3312773993527,5cyclictest3410901-21/usr/sbin/munin10:50:011
3312773993527,5cyclictest3410901-21/usr/sbin/munin10:50:011
331277399352,23cyclictest3365791-21ssh09:36:121
3312773993521,8cyclictest0-21swapper/110:21:101
3312773993521,8cyclictest0-21swapper/110:21:101
3312772993529,4cyclictest3335293-21kworker/u8:11-rpciod09:10:010
3312772993527,5cyclictest3330783-21systemd-run08:10:010
3312772993525,7cyclictest3375510-21unixbench_multi09:51:160
3312772993524,8cyclictest3316316-21fschecks_time07:20:570
3312774993428,3cyclictest0-21swapper/209:50:012
3312774993426,5cyclictest3367474-21ssh09:40:012
3312774993426,5cyclictest3335265-21munin-plugin-st08:25:002
3312774993426,5cyclictest3335265-21munin-plugin-st08:25:002
3312774993423,8cyclictest3342116-21ntp_kernel_err08:46:032
3312773993428,4cyclictest0-21swapper/108:20:011
3312773993426,5cyclictest421-21dbus-daemon11:25:011
3312773993426,5cyclictest421-21dbus-daemon11:25:011
3312773993426,5cyclictest3341284-21systemd-run08:45:011
331277299347,23cyclictest428-21mta-sts-daemon08:51:040
3312772993426,5cyclictest2524656-21cups-browsed12:21:510
3312772993425,7cyclictest3453119-21latency11:56:010
3312772993425,5cyclictest151ktimers/008:36:030
3312772993424,6cyclictest0-21swapper/010:30:010
3312772993424,6cyclictest0-21swapper/010:30:010
3312772993422,9cyclictest3341143-21unixbench_multi08:41:160
3312772993411,20cyclictest261-21systemd-journal07:35:010
3312774993326,3cyclictest0-21swapper/209:50:592
3312774993326,3cyclictest0-21swapper/209:11:042
3312774993325,5cyclictest534-21NetworkManager11:00:042
3312774993323,7cyclictest3326266-21sh07:55:012
331277399333,23cyclictest3450204-21ssh11:51:051
331277399333,23cyclictest3450204-21ssh11:51:051
3312773993325,5cyclictest3324701-21head07:46:171
3312773993325,5cyclictest3324701-21head07:46:171
3312773993320,9cyclictest420-21avahi-daemon10:56:411
3312772993326,5cyclictest3460235-21sh12:06:190
3312772993325,5cyclictest2524652-21cupsd07:36:030
3312772993323,7cyclictest3390866-21ls10:16:190
3312772993323,7cyclictest3390866-21ls10:16:190
3312772993321,3cyclictest171rcu_preempt07:51:090
3312772993315,14cyclictest12350irq/32-eth011:11:110
331277299330,6cyclictest3417818-21kworker/0:2+events_freezable_power_11:22:080
331277299330,6cyclictest3417818-21kworker/0:2+events_freezable_power_11:22:080
331277399323,9cyclictest171rcu_preempt09:31:071
3312773993225,4cyclictest0-21swapper/109:45:011
3312773993224,4cyclictest0-21swapper/109:10:021
3312773993221,8cyclictest3321074-21needreboot07:36:021
331277399321,5cyclictest171rcu_preempt12:31:061
3312772993225,3cyclictest0-21swapper/012:36:120
3312772993224,6cyclictest3329261-21cron08:05:000
3312772993224,5cyclictest3450819-21ssh11:51:170
3312772993224,5cyclictest3450819-21ssh11:51:170
3312772993218,3cyclictest171rcu_preempt07:56:100
331277299321,24cyclictest3465899-21rm12:16:130
3312774993125,3cyclictest0-21swapper/212:40:012
3312774993124,3cyclictest0-21swapper/210:21:112
3312774993124,3cyclictest0-21swapper/210:21:112
3312774993123,5cyclictest3455182-21ssh12:00:002
3312774993122,5cyclictest3337532-21missed_timers08:31:022
331277399316,20cyclictest3374901-21ssh09:51:031
331277399313,22cyclictest3468921-21ssh12:21:071
3312773993125,3cyclictest0-21swapper/107:20:001
3312773993123,5cyclictest3404676-21latency_hist10:40:001
3312773993123,5cyclictest3404676-21latency_hist10:40:001
3312773993123,5cyclictest0-21swapper/111:35:001
3312773993123,5cyclictest0-21swapper/111:35:001
331277399311,25cyclictest3359324-21ssh09:26:021
331277299313,21cyclictest3365576-21ssh09:36:040
3312772993124,4cyclictest3342777-21systemd-run08:50:010
3312772993122,5cyclictest151ktimers/008:31:050
331277299312,23cyclictest3405942-21ssh10:41:060
331277299312,22cyclictest3350022-21sh09:11:050
331277499304,20cyclictest3381265-21ssh10:01:092
3312774993023,4cyclictest3345748-21grep09:00:012
3312774993023,4cyclictest3321797-21kworker/u8:10-xprtiod07:45:002
3312774993023,4cyclictest1056-21mta-sts-daemon12:11:082
3312774993022,5cyclictest3468850-21ssh12:21:042
3312773993024,3cyclictest0-21swapper/111:45:061
3312773993024,3cyclictest0-21swapper/111:45:061
3312773993022,5cyclictest3462262-21ssh12:11:011
3312773993022,5cyclictest3452941-21gltestperf11:55:581
331277299305,14cyclictest191rcuc/010:12:000
3312772993024,3cyclictest3355050-21kworker/u8:7-xprtiod10:06:100
3312772993022,4cyclictest0-21swapper/010:33:380
3312772993022,4cyclictest0-21swapper/010:33:380
3312772993021,6cyclictest3440229-21tail11:35:540
3312772993021,6cyclictest3440229-21tail11:35:540
3312772993018,9cyclictest0-21swapper/009:15:560
331277499293,16cyclictest1056-21mta-sts-daemon11:31:052
331277499293,16cyclictest1056-21mta-sts-daemon11:31:052
3312774992923,3cyclictest0-21swapper/211:10:012
3312774992923,3cyclictest0-21swapper/210:26:022
3312774992923,3cyclictest0-21swapper/210:26:022
3312774992918,8cyclictest3316653-21ntp_kernel_pll_07:21:042
3312773992922,4cyclictest0-21swapper/108:15:011
331277399292,21cyclictest3350074-21rm09:11:071
3312773992921,5cyclictest439-21systemd-logind09:00:011
3312773992919,7cyclictest3424180-21systemd-run11:10:011
331277299292,20cyclictest3434850-21ssh11:26:110
3312772992913,11cyclictest3422282-21ssh11:06:050
3312774992822,3cyclictest0-21swapper/210:30:522
3312774992822,3cyclictest0-21swapper/210:30:522
3312774992821,4cyclictest3321070-21needreboot07:36:022
3312774992821,4cyclictest0-21swapper/208:15:002
3312774992821,3cyclictest0-21swapper/208:45:012
3312774992821,3cyclictest0-21swapper/208:20:002
3312774992820,5cyclictest3346639-21ntp_kernel_err09:01:032
3312774992820,5cyclictest1051-21mta-sts-daemon08:51:042
331277499281,16cyclictest3390417-21ntpq10:16:072
331277499281,16cyclictest3390417-21ntpq10:16:072
331277399283,14cyclictest428-21mta-sts-daemon08:06:051
3312773992822,3cyclictest0-21swapper/110:27:221
3312773992822,3cyclictest0-21swapper/110:27:221
3312773992821,4cyclictest1051-21mta-sts-daemon10:12:111
3312773992820,5cyclictest3335127-21unixbench-2d08:21:161
3312773992820,5cyclictest3335127-21unixbench-2d08:21:161
3312773992819,6cyclictest3317290-21systemd-run07:25:011
331277299285,2cyclictest171rcu_preempt07:15:580
3312772992819,5cyclictest0-21swapper/010:36:050
3312772992819,5cyclictest0-21swapper/010:36:050
3312772992818,3cyclictest3335269-21grep08:25:000
3312772992818,3cyclictest3335269-21grep08:25:000
3312772992815,10cyclictest3379896-21ssh10:00:010
3312772992814,11cyclictest0-21swapper/009:00:010
3312774992721,3cyclictest0-21swapper/207:20:002
331277399272,19cyclictest3353088-21ntpq09:16:041
3312773992718,6cyclictest439-21systemd-logind07:45:011
3312773992716,8cyclictest3314258-21munin-plugin-st07:15:001
3312772992712,10cyclictest421-21dbus-daemon12:27:540
3312774992618,5cyclictest3329292-21systemd-run08:05:012
331277399262,18cyclictest2524656-21cups-browsed08:01:111
3312773992618,5cyclictest624-21lldpd07:56:231
3312773992618,5cyclictest421-21dbus-daemon08:46:431
3312773992617,6cyclictest3338255-21munin-plugin-st08:35:001
331277299261,4cyclictest0-21swapper/008:26:060
3312774992518,4cyclictest1055-21mta-sts-daemon07:46:082
3312774992518,4cyclictest1055-21mta-sts-daemon07:46:082
3312773992517,5cyclictest3317608-21aten_r2power_fr07:25:541
331277299259,9cyclictest14-21ksoftirqd/007:46:040
331277299259,9cyclictest14-21ksoftirqd/007:46:040
3312772992519,4cyclictest3312771-21cyclictest09:03:030
3312772992513,10cyclictest0-21swapper/007:10:590
331277299249,9cyclictest3314242-21kworker/u8:4+events_unbound07:25:570
3312772992416,6cyclictest3323263-21latency_hist07:45:010
3312774992317,3cyclictest0-21swapper/208:00:012
331277399233,13cyclictest428-21mta-sts-daemon07:31:091
331277499222,15cyclictest3318149-21ntpq07:26:032
331277499221,16cyclictest3313786-21ps07:11:112
331277399222,15cyclictest3346729-21ntp_states09:01:081
3312773992216,3cyclictest0-21swapper/108:50:541
3312773992216,3cyclictest0-21swapper/107:10:011
331277599172,9cyclictest0-21swapper/311:45:033
331277599172,9cyclictest0-21swapper/311:45:033
331277499176,8cyclictest261-21systemd-journal07:10:012
331277599162,7cyclictest0-21swapper/309:31:043
331277599161,8cyclictest0-21swapper/310:51:093
331277599161,8cyclictest0-21swapper/310:51:093
331277599153,7cyclictest0-21swapper/309:41:073
331277599153,6cyclictest0-21swapper/312:31:203
331277599152,7cyclictest0-21swapper/312:26:083
331277599152,7cyclictest0-21swapper/312:01:033
331277599152,6cyclictest0-21swapper/310:21:093
331277599152,6cyclictest0-21swapper/310:21:093
331277599151,8cyclictest0-21swapper/312:06:073
331277599151,8cyclictest0-21swapper/308:21:073
331277599151,8cyclictest0-21swapper/308:21:073
331277599151,7cyclictest0-21swapper/309:46:053
331277599151,7cyclictest0-21swapper/309:16:073
331277599151,6cyclictest0-21swapper/307:16:043
331277599143,7cyclictest0-21swapper/311:15:583
331277599143,7cyclictest0-21swapper/311:01:033
331277599142,8cyclictest0-21swapper/310:16:073
331277599142,8cyclictest0-21swapper/310:16:073
331277599142,7cyclictest0-21swapper/312:16:133
331277599142,7cyclictest0-21swapper/310:31:023
331277599142,7cyclictest0-21swapper/310:31:023
331277599142,6cyclictest0-21swapper/311:56:063
331277599142,5cyclictest0-21swapper/309:35:193
331277599141,7cyclictest0-21swapper/311:31:063
331277599141,7cyclictest0-21swapper/311:31:063
331277599141,7cyclictest0-21swapper/311:26:073
331277599141,7cyclictest0-21swapper/311:21:073
331277599141,7cyclictest0-21swapper/311:21:073
331277599141,7cyclictest0-21swapper/310:26:083
331277599141,7cyclictest0-21swapper/310:26:083
331277599141,7cyclictest0-21swapper/309:56:043
331277599141,7cyclictest0-21swapper/309:01:033
331277599141,7cyclictest0-21swapper/307:46:063
331277599141,7cyclictest0-21swapper/307:46:063
331277599141,6cyclictest0-21swapper/308:01:073
331277599141,5cyclictest0-21swapper/307:21:033
331277599132,7cyclictest0-21swapper/311:36:043
331277599132,7cyclictest0-21swapper/311:36:043
331277599132,6cyclictest0-21swapper/312:36:103
331277599132,6cyclictest0-21swapper/312:11:073
331277599132,6cyclictest0-21swapper/311:06:043
331277599131,7cyclictest0-21swapper/311:51:083
331277599131,7cyclictest0-21swapper/311:51:083
331277599131,7cyclictest0-21swapper/310:56:053
331277599131,7cyclictest0-21swapper/309:11:053
331277599131,7cyclictest0-21swapper/308:56:073
331277599131,6cyclictest0-21swapper/311:11:053
331277599131,6cyclictest0-21swapper/308:41:063
331277599131,6cyclictest0-21swapper/308:36:073
331277599131,6cyclictest0-21swapper/308:26:053
331277599131,6cyclictest0-21swapper/308:11:033
331277599131,5cyclictest0-21swapper/307:11:033
331277599125,5cyclictest0-21swapper/311:45:053
331277599125,5cyclictest0-21swapper/311:45:053
331277599122,6cyclictest0-21swapper/310:46:173
331277599122,6cyclictest0-21swapper/310:46:173
331277599122,5cyclictest0-21swapper/309:20:573
331277599121,7cyclictest0-21swapper/307:41:093
331277599121,6cyclictest0-21swapper/310:01:043
331277599121,6cyclictest0-21swapper/309:51:033
331277599121,6cyclictest0-21swapper/309:26:023
331277599121,6cyclictest0-21swapper/308:46:053
331277599121,6cyclictest0-21swapper/308:31:083
331277599121,6cyclictest0-21swapper/307:36:023
331277599121,6cyclictest0-21swapper/307:31:023
331277599121,5cyclictest0-21swapper/308:51:043
331277599121,5cyclictest0-21swapper/307:56:053
331277599114,5cyclictest0-21swapper/310:41:113
331277599112,5cyclictest0-21swapper/312:21:073
331277599111,6cyclictest0-21swapper/310:35:593
331277599111,6cyclictest0-21swapper/310:35:593
331277599111,6cyclictest0-21swapper/309:06:053
331277599111,6cyclictest0-21swapper/307:51:053
331277599102,5cyclictest0-21swapper/310:10:543
331277599101,6cyclictest0-21swapper/308:06:043
331277599101,5cyclictest0-21swapper/310:05:373
33127759993,4cyclictest0-21swapper/308:16:053
33127759991,5cyclictest0-21swapper/307:25:533
33127759971,3cyclictest0-21swapper/307:10:003
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional