You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-12 - 14:51

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Wed Nov 12, 2025 12:39:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
110996995545,6cyclictest3438036-21cups-browsed11:04:062
110996995041,6cyclictest422-21dbus-daemon09:35:022
110994995042,5cyclictest422-21dbus-daemon09:30:010
110994995039,4cyclictest0-21swapper/009:14:060
110996994738,6cyclictest139505-21systemd-run08:45:012
110995994734,5cyclictest422-21dbus-daemon12:30:021
110995994734,5cyclictest422-21dbus-daemon12:30:021
110995994635,7cyclictest422-21dbus-daemon11:49:051
110996994536,5cyclictest0-21swapper/210:04:122
110995994537,5cyclictest237853-21latency_hist11:05:021
110994994538,4cyclictest313517-21munin-run12:40:010
110996994336,3cyclictest0-21swapper/210:50:022
110996994334,6cyclictest117007-21systemd-run07:30:012
110996994333,7cyclictest453-21systemd-logind09:30:012
110995994334,6cyclictest285693-21ssh12:05:011
110994994234,5cyclictest244223-21fschecks_count11:13:590
110996994134,4cyclictest0-21swapper/209:00:012
110996994133,5cyclictest120535-21fschecks_time07:43:592
110996994133,5cyclictest120535-21fschecks_time07:43:592
110995994133,5cyclictest241786-21systemd-run11:10:021
110995994133,5cyclictest117007-21systemd-run07:30:011
11099499411,32cyclictest304655-21ssh12:29:120
11099499411,32cyclictest304655-21ssh12:29:120
110996994032,5cyclictest249714-21ssh11:20:012
110996994031,6cyclictest245671-21cron11:15:012
110996994031,5cyclictest174105-21ssh09:45:012
110996994029,7cyclictest421-21avahi-daemon11:30:012
110995994034,3cyclictest0-21swapper/109:24:001
110995994032,5cyclictest281661-21idleruntime-cro12:00:001
110995994032,5cyclictest281661-21idleruntime-cro12:00:001
110994994032,5cyclictest237762-21cron11:05:010
11099499402,29cyclictest12450irq/32-eth012:14:100
110996993931,5cyclictest0-21swapper/209:24:232
110995993931,5cyclictest114033-21latency_hist07:20:021
110995993931,5cyclictest114033-21latency_hist07:20:021
110995993930,6cyclictest111520-21uniq07:14:001
110995993930,6cyclictest111520-21uniq07:14:001
110994993931,5cyclictest224774-21ntp_kernel_pll_10:49:080
110994993929,7cyclictest3438036-21cups-browsed09:54:170
110996993830,5cyclictest301200-21unixbench_singl12:24:182
110996993829,6cyclictest289303-21head12:09:192
110996993829,6cyclictest221009-21mailstats10:44:132
110996993829,5cyclictest955-21mta-sts-daemon10:14:082
110996993827,7cyclictest421-21avahi-daemon12:25:162
110996993827,7cyclictest421-21avahi-daemon12:25:162
110995993830,5cyclictest127510-21systemd-run08:05:021
110995993829,6cyclictest214043-21latency_hist10:35:021
110995993829,6cyclictest169612-21timerwakeupswit09:39:191
110995993829,6cyclictest169612-21timerwakeupswit09:39:191
110995993829,6cyclictest0-21swapper/111:39:061
110994993831,5cyclictest217812-21cut10:40:010
110994993830,5cyclictest166228-21ssh09:35:020
110994993828,5cyclictest0-21swapper/010:04:090
110996993730,4cyclictest125956-21cron08:00:012
110996993729,5cyclictest142401-21unixbench_singl08:54:192
110996993728,6cyclictest133857-21basename08:28:582
110996993727,7cyclictest3438036-21cups-browsed09:59:042
110995993729,5cyclictest162448-21basename09:30:011
110995993729,5cyclictest135515-21fschecks_count08:34:001
110994993732,2cyclictest0-21swapper/010:24:230
110994993731,4cyclictest277663-21/usr/sbin/munin11:55:020
110994993731,4cyclictest277663-21/usr/sbin/munin11:55:020
110994993731,3cyclictest151ktimers/011:29:130
110994993730,5cyclictest12450irq/32-eth009:24:080
110994993730,4cyclictest201968-21systemd-run10:20:010
110996993628,5cyclictest292065-21aten_r2power_vo12:13:572
110996993626,7cyclictest0-21swapper/209:18:582
110996993625,7cyclictest3438036-21cups-browsed07:54:122
110996993624,9cyclictest253182-21timerwakeupswit11:24:182
11099599365,25cyclictest216780-21ssh10:39:081
110995993629,4cyclictest233790-21grep11:00:011
110995993628,5cyclictest142502-21latency_hist08:55:011
110995993627,6cyclictest139371-21unixbench_multi08:44:181
110995993626,7cyclictest421-21avahi-daemon12:08:431
11099499369,16cyclictest151ktimers/008:29:080
11099499363,27cyclictest184998-21ssh09:59:130
11099499363,27cyclictest169075-21ssh09:39:060
11099499363,27cyclictest169075-21ssh09:39:060
110994993628,4cyclictest174249-21latency_hist09:45:020
110994993616,14cyclictest12450irq/32-eth010:34:120
110996993528,4cyclictest135503-21kworker/u8:13-xprtiod12:34:062
110996993527,5cyclictest181986-21systemd-run09:55:012
110996993523,9cyclictest280547-21needreboot11:59:072
110996993523,9cyclictest280547-21needreboot11:59:072
110995993528,5cyclictest110993-21cyclictest10:09:051
110995993527,5cyclictest185979-21basename10:00:011
110995993526,6cyclictest277780-21latency_hist11:55:031
110995993526,6cyclictest277780-21latency_hist11:55:031
110995993526,6cyclictest257706-21systemd-run11:30:011
110995993526,5cyclictest178237-21kworker/u8:1+xprtiod09:54:071
110995993526,5cyclictest0-21swapper/109:14:071
110994993527,5cyclictest265024-21timerandwakeup11:39:180
110994993522,9cyclictest151ktimers/010:14:080
110994993513,18cyclictest280764-21ssh11:59:150
110994993513,18cyclictest280764-21ssh11:59:150
110996993429,2cyclictest0-21swapper/211:44:012
110996993427,4cyclictest959-21mta-sts-daemon10:36:052
110996993427,4cyclictest228585-21missed_timers10:54:022
110996993425,6cyclictest3438036-21cups-browsed11:52:012
110996993425,6cyclictest3438036-21cups-browsed11:52:012
110995993426,5cyclictest144004-21systemd-run09:00:021
110995993426,5cyclictest132020-21systemd-run08:20:021
110995993425,6cyclictest0-21swapper/111:14:091
110995993425,5cyclictest220344-21fschecks_count10:43:591
110995993413,7cyclictest59950lldpd11:43:361
110994993428,3cyclictest0-21swapper/011:44:080
110994993425,7cyclictest233112-21switchtime10:59:160
110996993325,5cyclictest233775-21ssh11:00:012
110996993323,7cyclictest150059-21ntpq09:14:102
110996993319,7cyclictest193970-21latency_hist10:10:022
110995993326,4cyclictest196810-21needreboot10:14:061
110995993325,5cyclictest313095-21unixbench_multi12:39:231
110995993325,5cyclictest146973-21diskmemload11:34:031
110995993325,5cyclictest121508-21systemd-run07:45:011
110995993325,5cyclictest121508-21systemd-run07:45:011
110995993325,4cyclictest120004-21sendmail07:40:011
11099599332,23cyclictest248770-21ntpq11:19:121
11099499339,14cyclictest151ktimers/007:19:080
11099499339,14cyclictest151ktimers/007:19:080
110994993326,4cyclictest177607-21sed09:49:170
11099499332,25cyclictest272587-21ssh11:49:080
110994993316,11cyclictest192897-21ssh10:09:060
110994993314,13cyclictest3438002-21cupsd12:09:060
110996993226,3cyclictest0-21swapper/207:49:082
110996993224,5cyclictest959-21mta-sts-daemon10:23:092
110996993224,5cyclictest209878-21ssh10:30:012
110996993223,5cyclictest453-21systemd-logind07:35:012
110996993222,6cyclictest131987-21systemctl08:20:012
11099699322,24cyclictest265300-21ssh11:39:212
110996993222,4cyclictest0-21swapper/209:35:452
110996993222,4cyclictest0-21swapper/209:35:452
11099699321,28cyclictest272661-21ssh11:49:112
11099599323,24cyclictest208906-21ssh10:29:101
110995993227,2cyclictest0-21swapper/108:40:011
110995993226,3cyclictest0-21swapper/112:15:011
110995993224,5cyclictest122979-21munin-plugin-st07:50:001
110995993223,6cyclictest266-21systemd-journal11:25:011
110995993223,4cyclictest0-21swapper/112:24:311
110995993222,5cyclictest957-21mta-sts-daemon10:54:081
110994993225,5cyclictest141519-21fschecks_time08:54:000
110994993225,4cyclictest297713-21ssh12:20:010
110994993224,4cyclictest0-21swapper/010:29:120
110994993223,5cyclictest151ktimers/012:04:070
110994993220,10cyclictest128021-21fschecks_count08:09:020
11099699313,22cyclictest260598-21ssh11:34:062
110996993125,3cyclictest0-21swapper/210:19:022
110996993123,5cyclictest959-21mta-sts-daemon12:03:442
110996993123,5cyclictest130351-21unixbench-2d08:14:202
11099699312,24cyclictest146973-21diskmemload12:19:082
11099699312,21cyclictest312478-21ssh12:39:092
110996993121,6cyclictest0-21swapper/209:10:032
110995993125,3cyclictest0-21swapper/112:30:461
110995993123,5cyclictest3438036-21cups-browsed10:23:171
110995993123,5cyclictest134881-21unixbench_multi08:29:201
110995993123,5cyclictest125913-21unixbench_singl07:59:221
110995993123,4cyclictest0-21swapper/108:24:101
11099599312,23cyclictest153970-21diskmemload09:19:151
110995993120,8cyclictest176819-21missed_timers09:49:011
110994993124,4cyclictest221829-21sh10:45:010
110994993123,6cyclictest307787-21aten_r2power_cu12:33:550
11099699306,6cyclictest0-21swapper/208:39:082
110996993021,6cyclictest132534-21fschecks_count08:24:002
11099599305,21cyclictest173030-21ntpq09:44:061
110995993024,3cyclictest0-21swapper/110:05:011
11099599302,22cyclictest165189-21ssh09:34:071
110994993024,4cyclictest253288-21unixbench_singl11:24:190
110994993023,4cyclictest143965-21grep09:00:010
110994993020,8cyclictest229258-21timerwakeupswit10:54:180
11099499301,23cyclictest260714-21ssh11:34:100
11099699295,8cyclictest171rcu_preempt08:04:112
110996992922,4cyclictest0-21swapper/211:09:002
110996992922,3cyclictest0-21swapper/209:49:122
110996992921,4cyclictest0-21swapper/210:33:582
110995992922,5cyclictest297713-21ssh12:20:011
110995992922,4cyclictest124432-21unixbench_singl07:54:191
110995992921,5cyclictest146917-21head09:09:201
110994992923,3cyclictest240518-21latency11:09:040
110994992922,5cyclictest3438036-21cups-browsed11:16:150
110994992922,5cyclictest137986-21sendmail08:40:010
110994992918,8cyclictest153186-21aten_r2power_vo09:18:580
110994992918,5cyclictest0-21swapper/008:19:140
11099599282,20cyclictest224847-21ssh10:49:101
110995992820,5cyclictest128577-21systemctl08:09:151
11099599281,23cyclictest201062-21ssh10:19:161
110994992822,4cyclictest959-21mta-sts-daemon08:14:000
110994992821,4cyclictest126531-21fschecks_count08:04:000
110994992821,4cyclictest115486-21cron07:25:010
110994992819,7cyclictest111808-21needreboot07:14:050
110994992819,7cyclictest111808-21needreboot07:14:050
110994992813,9cyclictest12450irq/32-eth012:24:100
11099499281,21cyclictest657-21snmpd07:59:120
110996992719,5cyclictest135541-21gltestperf08:34:012
110995992721,3cyclictest0-21swapper/107:35:011
110994992721,4cyclictest136437-21unixbench_singl08:34:190
110994992721,3cyclictest956-21mta-sts-daemon08:44:030
110994992719,5cyclictest421-21avahi-daemon07:47:130
110996992617,7cyclictest113373-21ntp_kernel_pll_07:19:082
110996992617,7cyclictest113373-21ntp_kernel_pll_07:19:082
110996992614,9cyclictest140341-21ntp_kernel_err08:49:032
110996992614,9cyclictest140341-21ntp_kernel_err08:49:032
110995992620,3cyclictest0-21swapper/107:24:181
110995992619,5cyclictest0-21swapper/108:48:541
110995992619,5cyclictest0-21swapper/108:48:541
110994992621,3cyclictest110993-21cyclictest07:54:030
110994992620,3cyclictest0-21swapper/009:10:010
110994992620,3cyclictest0-21swapper/009:04:190
110994992617,7cyclictest121478-21cron07:45:010
110994992617,7cyclictest121478-21cron07:45:010
110994992617,5cyclictest151ktimers/007:34:040
110996992518,4cyclictest119938-21unixbench_singl07:39:182
110996992518,3cyclictest0-21swapper/208:09:202
110996992513,6cyclictest3438036-21cups-browsed07:14:122
110996992513,6cyclictest3438036-21cups-browsed07:14:122
110996992417,4cyclictest959-21mta-sts-daemon09:04:092
110994992412,9cyclictest0-21swapper/007:30:010
110995992316,4cyclictest103118-21kworker/u8:4-xprtiod07:10:031
110995992315,6cyclictest140988-21kworker/u8:5+xprtiod09:05:001
110994992316,4cyclictest3438036-21cups-browsed08:24:520
110996992211,7cyclictest266-21systemd-journal07:23:472
110995992216,3cyclictest0-21swapper/108:14:031
110994992211,8cyclictest422-21dbus-daemon07:10:010
11099799218,5cyclictest0-21swapper/311:49:153
110994992116,2cyclictest0-21swapper/007:40:010
110994992114,4cyclictest3438036-21cups-browsed08:46:310
110994992114,4cyclictest3438036-21cups-browsed08:46:310
11099799208,5cyclictest0-21swapper/312:29:083
11099799208,5cyclictest0-21swapper/312:29:083
11099799198,4cyclictest0-21swapper/308:19:113
11099799188,5cyclictest0-21swapper/308:54:103
11099799188,4cyclictest0-21swapper/307:49:063
11099799187,6cyclictest0-21swapper/310:14:073
11099799187,5cyclictest0-21swapper/309:29:083
11099799179,5cyclictest0-21swapper/311:44:083
11099799178,5cyclictest0-21swapper/311:39:083
11099799178,5cyclictest0-21swapper/310:34:083
11099799178,4cyclictest0-21swapper/309:19:073
11099799178,4cyclictest0-21swapper/307:59:103
11099799178,4cyclictest0-21swapper/307:29:073
11099799177,5cyclictest0-21swapper/310:29:073
11099799177,5cyclictest0-21swapper/309:24:073
11099799169,5cyclictest0-21swapper/310:49:073
11099799169,4cyclictest0-21swapper/311:29:053
11099799169,4cyclictest0-21swapper/311:09:073
11099799169,4cyclictest0-21swapper/310:59:063
11099799169,4cyclictest0-21swapper/307:24:083
11099799168,5cyclictest0-21swapper/312:19:053
11099799168,5cyclictest0-21swapper/308:24:063
11099799168,4cyclictest0-21swapper/310:54:083
11099799168,4cyclictest0-21swapper/310:19:123
11099799168,4cyclictest0-21swapper/310:04:083
11099799168,4cyclictest0-21swapper/309:34:053
11099799168,4cyclictest0-21swapper/309:04:053
11099799168,4cyclictest0-21swapper/308:59:113
11099799168,4cyclictest0-21swapper/308:09:133
11099799168,4cyclictest0-21swapper/308:04:123
11099799168,4cyclictest0-21swapper/307:54:083
11099799168,4cyclictest0-21swapper/307:44:043
11099799168,4cyclictest0-21swapper/307:44:043
11099799167,5cyclictest0-21swapper/312:39:093
11099799167,5cyclictest0-21swapper/312:30:233
11099799167,5cyclictest0-21swapper/312:24:013
11099799167,5cyclictest0-21swapper/310:44:053
11099799167,5cyclictest0-21swapper/310:09:023
11099799167,5cyclictest0-21swapper/309:52:003
11099799167,5cyclictest0-21swapper/309:14:063
11099799167,5cyclictest0-21swapper/307:19:083
11099799167,5cyclictest0-21swapper/307:19:083
11099799166,6cyclictest0-21swapper/311:15:303
11099799162,9cyclictest0-21swapper/311:14:093
11099699162,11cyclictest110949-21/usr/sbin/munin07:10:012
11099799159,4cyclictest0-21swapper/308:39:083
11099799158,5cyclictest0-21swapper/309:42:283
11099799158,4cyclictest0-21swapper/311:24:093
11099799158,4cyclictest0-21swapper/311:01:433
11099799158,4cyclictest0-21swapper/310:37:153
11099799158,4cyclictest0-21swapper/309:49:043
11099799158,4cyclictest0-21swapper/308:49:033
11099799158,4cyclictest0-21swapper/308:49:033
11099799158,4cyclictest0-21swapper/308:29:143
11099799158,4cyclictest0-21swapper/307:39:073
11099799157,5cyclictest0-21swapper/312:08:353
11099799157,5cyclictest0-21swapper/311:58:593
11099799157,5cyclictest0-21swapper/311:58:593
11099799157,5cyclictest0-21swapper/309:59:143
11099799157,5cyclictest0-21swapper/307:34:083
11099799157,4cyclictest0-21swapper/312:04:063
11099799157,4cyclictest0-21swapper/309:35:433
11099799157,4cyclictest0-21swapper/309:35:433
11099799156,6cyclictest0-21swapper/310:24:103
11099799155,6cyclictest0-21swapper/311:50:383
11099799155,6cyclictest0-21swapper/311:50:383
110997991511,2cyclictest0-21swapper/307:10:013
11099799148,4cyclictest0-21swapper/307:14:093
11099799148,4cyclictest0-21swapper/307:14:093
11099799147,4cyclictest0-21swapper/312:10:093
11099799147,4cyclictest0-21swapper/311:30:103
11099799147,4cyclictest0-21swapper/308:33:573
11099799147,4cyclictest0-21swapper/308:14:043
11099799146,5cyclictest0-21swapper/308:43:593
11099799138,3cyclictest0-21swapper/309:09:073
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional