You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-16 - 04:36

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Wed Sep 16, 2026 00:42:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2263076995545,6cyclictest2272093-21systemd-run19:40:012
226307599533,7cyclictest2370626-21ssh23:06:171
2263076995040,6cyclictest2278088-21systemd-run20:00:012
2263075994930,16cyclictest2324782-21uname21:51:131
226307599483,8cyclictest2277511-21ntpq19:56:171
2263075994832,12cyclictest0-21swapper/121:10:011
2263074994843,3cyclictest151ktimers/023:36:180
2263074994843,3cyclictest151ktimers/023:36:180
2263075994735,8cyclictest575222-21cups-browsed22:21:131
226307599464,14cyclictest2346350-21ntpq22:26:181
226307599461,12cyclictest2280491-21ntpq20:06:151
226307599452,14cyclictest2312709-21ntpq21:31:181
2263076994436,5cyclictest2345950-21sh22:26:082
2263075994436,5cyclictest2317478-21cron21:40:001
2263075994433,8cyclictest2275074-21sh19:50:011
2263075994421,19cyclictest0-21swapper/123:15:031
226307699431,38cyclictest2349292-21sh22:31:142
226307599432,5cyclictest2283524-21ntpq20:16:151
226307599431,12cyclictest2264529-21sshd19:12:101
2263076994234,4cyclictest0-21swapper/200:00:012
2263075994233,5cyclictest2317744-21kworker/u8:8-xprtiod22:11:181
2263075994223,16cyclictest0-21swapper/122:56:261
2263075994218,20cyclictest575222-21cups-browsed21:43:031
2263075994218,19cyclictest0-21swapper/100:21:261
2263075994218,19cyclictest0-21swapper/100:21:261
226307599421,11cyclictest2389005-21ntpq23:36:201
226307599421,11cyclictest2389005-21ntpq23:36:201
2263074994235,4cyclictest2402927-21awk00:00:010
2263076994135,3cyclictest0-21swapper/222:40:012
2263076994133,5cyclictest2303559-21proc_pri21:16:202
2263075994133,5cyclictest2357156-21ssh22:45:011
2263075994133,5cyclictest2357156-21ssh22:45:011
2263075994133,4cyclictest1002-21mta-sts-daemon21:46:391
226307599412,8cyclictest2306464-21ntpq21:21:131
2263075994114,23cyclictest2339390-21/usr/sbin/munin22:16:051
226307599411,12cyclictest665-21snmpd23:31:111
2263074994132,6cyclictest2391571-21fschecks_time23:41:050
226307699403,29cyclictest2361471-21ssh22:51:132
226307699403,29cyclictest2361471-21ssh22:51:132
2263076994032,5cyclictest2294572-21systemd-run20:55:022
2263076994032,3cyclictest0-21swapper/221:41:142
2263076994026,7cyclictest2404189-21sh00:01:132
226307599403,21cyclictest2413346-21ssh00:16:121
226307599403,21cyclictest2413346-21ssh00:16:121
2263075994029,9cyclictest2291580-21sh20:45:011
2263075994029,9cyclictest2291580-21sh20:45:011
226307599402,11cyclictest2265468-21ntpq19:16:151
2263075994017,20cyclictest2341945-21kworker/u8:3-xprtiod22:51:111
2263075994017,20cyclictest2341945-21kworker/u8:3-xprtiod22:51:111
2263075994015,22cyclictest0-21swapper/123:16:121
2263075994015,22cyclictest0-21swapper/121:30:011
2263075994014,2cyclictest0-21swapper/122:01:061
226307599401,13cyclictest2334055-21ssh22:06:141
226307599401,12cyclictest2303640-21sendmail_mailtr21:16:211
2263074994033,4cyclictest1003-21mta-sts-daemon23:16:080
2263074994030,7cyclictest2387650-21cron23:35:000
2263076993932,4cyclictest2335832-21latency_hist22:10:012
2263076993931,5cyclictest2345010-21latency_hist22:25:022
2263076993931,5cyclictest2293079-21systemd-run20:50:012
2263076993930,6cyclictest420-21avahi-daemon23:36:292
2263076993930,6cyclictest420-21avahi-daemon23:36:292
226307599392,16cyclictest2352342-21ssh22:36:131
2263075993913,4cyclictest2288039-21ntpq20:31:181
2263075993913,3cyclictest0-21swapper/123:41:161
226307599391,14cyclictest2379714-21ntpq23:21:091
226307599391,11cyclictest2269950-21ntpq19:31:121
226307599390,13cyclictest2278996-21awk20:01:141
2263076993830,5cyclictest2412182-21latency_hist00:15:022
2263076993829,6cyclictest575222-21cups-browsed22:56:362
2263076993829,6cyclictest2287013-21head20:26:262
2263076993828,5cyclictest0-21swapper/222:41:102
2263076993828,5cyclictest0-21swapper/222:41:102
2263076993824,9cyclictest2395201-21wc23:46:202
2263075993829,5cyclictest2425549-21ntp_kernel_pll_00:36:131
226307599382,7cyclictest2407229-21ntpq00:06:111
226307599381,13cyclictest2402638-21ssh23:59:111
226307599381,13cyclictest2349266-21ntpq22:31:131
226307599381,13cyclictest2268561-21ntpq19:26:181
226307599381,12cyclictest2300263-21ssh21:11:111
2263074993830,5cyclictest2369959-21head23:06:030
2263074993828,7cyclictest0-21swapper/000:36:040
2263076993729,5cyclictest2329670-21cron22:00:002
2263076993729,5cyclictest2329670-21cron22:00:002
2263076993728,6cyclictest265-21systemd-journal20:25:012
2263076993728,6cyclictest2279587-21systemd-run20:05:012
226307699372,28cyclictest575219-21cupsd20:31:182
226307599372,13cyclictest2273042-21ntpq19:41:191
226307599372,10cyclictest428-21mta-sts-daemon19:21:121
226307599371,14cyclictest2297092-21ps21:01:221
226307599371,13cyclictest2394604-21rm23:46:051
226307599371,11cyclictest2398282-21/usr/sbin/munin23:51:191
2263074993731,3cyclictest0-21swapper/000:31:210
2263074993730,4cyclictest1003-21mta-sts-daemon21:30:290
2263074993729,6cyclictest2336668-21fschecks_time22:11:060
2263074993729,5cyclictest2315906-21sendmail_mailqu21:36:220
2263074993728,6cyclictest2300310-21ntp_kernel_pll_21:11:130
2263074993727,7cyclictest2346298-21ntp_offset22:26:160
2263076993628,5cyclictest2425759-21sendmail_mailqu00:36:202
2263076993628,5cyclictest2325386-21unixbench-2d21:51:282
2263076993628,4cyclictest2275588-21fschecks_time19:51:072
226307699362,27cyclictest2413403-21ssh00:16:142
226307699362,27cyclictest2413403-21ssh00:16:142
2263075993628,5cyclictest2382731-21grep23:26:101
226307599361,12cyclictest2367695-21ntpq23:01:161
226307599361,10cyclictest2289407-21sshd20:36:101
2263074993630,3cyclictest0-21swapper/000:05:020
2263074993625,8cyclictest2373505-21missed_timers23:11:090
2263074993620,12cyclictest0-21swapper/021:16:100
2263076993524,7cyclictest575219-21cupsd22:13:512
226307599356,7cyclictest2410188-21cat00:11:081
2263075993527,5cyclictest2271023-21cpuspeed19:36:041
226307599351,9cyclictest2299057-21diskmemload22:48:081
226307599351,9cyclictest2299057-21diskmemload22:48:081
2263075993512,2cyclictest0-21swapper/120:21:071
226307599351,11cyclictest2419191-21/usr/sbin/munin00:26:071
226307599351,11cyclictest2327432-21cat21:56:031
226307599351,11cyclictest2327432-21cat21:56:031
2263074993528,5cyclictest421-21dbus-daemon20:00:010
2263074993528,5cyclictest12650irq/32-eth000:16:160
2263074993528,5cyclictest12650irq/32-eth000:16:160
2263074993528,4cyclictest2363259-21idleruntime-cro22:55:000
2263074993528,4cyclictest2363259-21idleruntime-cro22:55:000
2263074993527,6cyclictest2319964-21sshd21:43:100
2263074993527,5cyclictest439-21systemd-logind19:10:020
2263074993514,17cyclictest151ktimers/020:26:150
226307699345,19cyclictest2416407-21ssh00:21:122
226307699345,19cyclictest2416407-21ssh00:21:122
2263076993428,3cyclictest0-21swapper/223:26:122
2263076993428,3cyclictest0-21swapper/221:10:012
2263076993426,4cyclictest2332774-21kworker/u8:6-rpciod22:17:432
2263076993426,4cyclictest2296044-21munin-plugin-st21:00:012
2263076993425,5cyclictest2310062-21unixbench-2d21:26:262
2263076993424,6cyclictest421-21dbus-daemon22:02:522
2263076993420,6cyclictest0-21swapper/223:40:202
226307599343,7cyclictest2422505-21ntpq00:31:151
226307599342,8cyclictest2404179-21ntpq00:01:121
2263074993427,4cyclictest2421276-21idleruntime-cro00:30:000
2263074993426,5cyclictest575222-21cups-browsed21:52:360
2263074993424,8cyclictest2306395-21needreboot21:21:110
2263074993422,8cyclictest12650irq/32-eth023:51:120
2263076993328,2cyclictest0-21swapper/223:20:012
2263076993326,4cyclictest2380290-21unixbench-2d23:21:232
2263076993326,4cyclictest0-21swapper/223:11:112
2263076993326,3cyclictest0-21swapper/221:46:182
2263076993325,5cyclictest485-21in:imuxsock00:10:012
2263076993325,5cyclictest421-21dbus-daemon22:50:012
2263076993325,5cyclictest421-21dbus-daemon22:50:012
2263075993325,4cyclictest1000-21mta-sts-daemon20:46:141
226307599331,13cyclictest2281974-21ntpq20:11:151
226307599331,10cyclictest2263086-21systemd-run19:10:011
2263074993325,6cyclictest2329684-21sendmail22:00:000
2263074993325,6cyclictest2329684-21sendmail22:00:000
2263074993322,8cyclictest0-21swapper/021:30:010
2263074993322,5cyclictest151ktimers/021:46:140
2263074993320,10cyclictest2292977-21unixbench_singl20:46:240
226307499331,26cyclictest151ktimers/022:46:200
226307499331,26cyclictest151ktimers/022:46:200
226307699326,16cyclictest428-21mta-sts-daemon21:36:152
226307699325,20cyclictest2263915-21sshd19:11:112
2263076993225,4cyclictest1000-21mta-sts-daemon00:31:222
2263076993224,5cyclictest2296600-21fschecks_time21:01:092
2263076993224,4cyclictest2380983-21kworker/u8:11-rpciod00:25:252
2263074993226,4cyclictest2395569-21unixbench_multi23:46:250
226307499322,27cyclictest2263073-21cyclictest22:06:130
2263076993125,3cyclictest0-21swapper/223:51:062
2263076993125,3cyclictest0-21swapper/219:45:012
2263076993123,5cyclictest2283090-21fschecks_time20:16:052
2263076993121,4cyclictest0-21swapper/223:31:172
226307499312,8cyclictest2352434-21ssh22:36:180
2263074993125,4cyclictest2344962-21cron22:25:010
2263074993124,4cyclictest2406994-21ssh00:06:070
2263074993122,4cyclictest12650irq/32-eth022:41:100
2263074993122,4cyclictest12650irq/32-eth022:41:100
2263074993112,14cyclictest12650irq/32-eth000:21:120
2263074993112,14cyclictest12650irq/32-eth000:21:120
2263074993110,16cyclictest665-21snmpd22:00:420
226307699303,12cyclictest2299057-21diskmemload23:03:492
2263076993023,3cyclictest0-21swapper/223:06:032
2263076993021,7cyclictest2280384-21needreboot20:06:102
2263075993023,4cyclictest2287080-21systemd-run20:30:011
226307599301,10cyclictest2275988-21grep19:51:161
2263074993022,5cyclictest2284097-21systemd-run20:20:010
2263074993021,7cyclictest2380104-21switchtime23:21:210
2263074993019,8cyclictest2273561-21cron19:45:010
2263076992923,3cyclictest0-21swapper/221:21:132
2263076992923,3cyclictest0-21swapper/221:11:112
2263076992922,4cyclictest1003-21mta-sts-daemon20:40:012
2263076992922,3cyclictest0-21swapper/219:20:012
226307499293,20cyclictest623-21sshd19:11:110
2263074992923,3cyclictest2340715-21unixbench_singl22:16:260
2263074992923,3cyclictest0-21swapper/000:11:140
2263074992921,6cyclictest2297556-21sh21:05:010
226307499291,9cyclictest171rcu_preempt20:01:110
2263074992918,8cyclictest2276484-21unixbench_singl19:51:250
2263074992916,10cyclictest421-21dbus-daemon19:16:150
226307499291,23cyclictest2367569-21ssh23:01:100
2263076992820,5cyclictest1-21systemd19:32:442
2263076992818,7cyclictest575222-21cups-browsed20:12:362
226307599285,8cyclictest575222-21cups-browsed20:55:531
226307499284,18cyclictest151ktimers/020:56:120
2263074992822,4cyclictest0-21swapper/023:26:250
2263074992820,6cyclictest2289084-21fschecks_time20:36:050
2263076992722,2cyclictest0-21swapper/220:41:242
2263076992722,2cyclictest0-21swapper/220:41:242
2263076992721,3cyclictest0-21swapper/221:31:022
2263074992721,4cyclictest2263073-21cyclictest19:31:080
2263074992721,3cyclictest0-21swapper/022:56:270
2263074992721,3cyclictest0-21swapper/020:12:190
2263074992714,6cyclictest2280092-21fschecks_count20:06:060
2263076992619,4cyclictest1002-21mta-sts-daemon19:21:112
2263074992620,3cyclictest0-21swapper/019:21:230
2263074992618,6cyclictest1003-21mta-sts-daemon19:26:190
226307499261,18cyclictest2349293-21sh22:31:140
2263074992518,5cyclictest1003-21mta-sts-daemon20:51:030
2263074992517,6cyclictest2288212-21switchtime20:31:230
2263076992416,5cyclictest575222-21cups-browsed19:25:362
226307499242,16cyclictest420-21avahi-daemon20:21:130
226307499232,15cyclictest2290568-21diskstats20:41:050
226307499232,15cyclictest2290568-21diskstats20:41:050
2263074992314,5cyclictest1001-21mta-sts-daemon21:06:180
2263076992215,4cyclictest1003-21mta-sts-daemon19:48:102
226307499221,9cyclictest2271837-21timerwakeupswit19:36:250
2263074992212,7cyclictest151ktimers/019:46:190
226307599202,14cyclictest958-21sendmail-mta20:51:171
226307699186,9cyclictest1-21systemd19:10:012
226307799174,6cyclictest0-21swapper/323:11:113
226307799174,5cyclictest0-21swapper/322:51:123
226307799174,5cyclictest0-21swapper/322:51:123
226307799172,8cyclictest0-21swapper/322:46:123
226307799172,8cyclictest0-21swapper/322:46:123
226307799164,5cyclictest0-21swapper/323:06:213
226307799162,7cyclictest0-21swapper/321:16:123
226307799161,8cyclictest0-21swapper/300:16:243
226307799161,8cyclictest0-21swapper/300:16:243
226307799161,6cyclictest0-21swapper/319:41:133
226307799153,6cyclictest0-21swapper/322:16:203
226307799153,6cyclictest0-21swapper/322:11:203
226307799153,5cyclictest0-21swapper/321:56:123
226307799153,5cyclictest0-21swapper/321:56:123
226307799152,7cyclictest0-21swapper/323:36:163
226307799152,7cyclictest0-21swapper/323:36:163
226307799152,6cyclictest0-21swapper/321:36:143
226307799152,5cyclictest0-21swapper/320:26:133
226307799151,7cyclictest0-21swapper/322:26:173
226307799145,6cyclictest0-21swapper/322:36:133
226307799143,7cyclictest0-21swapper/322:21:193
226307799142,7cyclictest0-21swapper/323:56:153
226307799142,7cyclictest0-21swapper/322:56:113
226307799142,5cyclictest0-21swapper/320:56:143
226307799141,8cyclictest0-21swapper/322:35:013
226307799141,7cyclictest0-21swapper/322:06:113
226307799141,7cyclictest0-21swapper/321:51:213
226307799141,7cyclictest0-21swapper/321:41:203
226307799141,6cyclictest0-21swapper/321:01:203
226307799141,6cyclictest0-21swapper/320:11:193
226307799141,6cyclictest0-21swapper/319:31:133
226307799141,6cyclictest0-21swapper/319:26:123
226307799141,6cyclictest0-21swapper/300:21:163
226307799141,6cyclictest0-21swapper/300:21:163
226307799137,4cyclictest0-21swapper/321:26:103
226307799133,6cyclictest0-21swapper/323:51:053
226307799133,4cyclictest0-21swapper/320:36:103
226307799133,4cyclictest0-21swapper/319:21:153
226307799132,6cyclictest0-21swapper/300:01:033
226307799131,7cyclictest0-21swapper/323:31:143
226307799131,7cyclictest0-21swapper/322:41:013
226307799131,7cyclictest0-21swapper/322:41:013
226307799131,7cyclictest0-21swapper/321:31:063
226307799131,5cyclictest0-21swapper/320:21:193
226307799126,4cyclictest0-21swapper/323:01:033
226307799126,4cyclictest0-21swapper/322:01:133
226307799123,5cyclictest0-21swapper/321:11:123
226307799123,5cyclictest0-21swapper/321:06:103
226307799122,6cyclictest0-21swapper/323:21:053
226307799122,5cyclictest0-21swapper/320:01:123
226307799121,6cyclictest0-21swapper/323:46:033
226307799121,6cyclictest0-21swapper/321:46:103
226307799121,6cyclictest0-21swapper/320:41:123
226307799121,6cyclictest0-21swapper/320:41:123
226307799121,6cyclictest0-21swapper/320:10:013
226307799121,6cyclictest0-21swapper/319:16:173
226307799121,6cyclictest0-21swapper/300:36:023
226307799116,3cyclictest0-21swapper/323:41:183
226307799116,3cyclictest0-21swapper/320:20:013
226307799116,3cyclictest0-21swapper/300:31:073
226307799115,4cyclictest0-21swapper/319:36:253
226307799113,4cyclictest0-21swapper/323:16:043
226307799112,5cyclictest0-21swapper/319:56:133
226307799112,5cyclictest0-21swapper/300:26:013
226307799111,6cyclictest0-21swapper/300:11:013
226307799111,6cyclictest0-21swapper/300:06:043
226307799111,5cyclictest0-21swapper/320:50:023
226307799111,5cyclictest0-21swapper/319:51:173
226307799101,6cyclictest0-21swapper/323:26:063
226307799101,5cyclictest0-21swapper/320:31:073
22630779994,3cyclictest0-21swapper/320:53:183
22630779992,5cyclictest421rcuc/319:10:013
22630779992,5cyclictest0-21swapper/321:21:143
22630779991,5cyclictest0-21swapper/319:11:103
22630779990,5cyclictest0-21swapper/319:46:183
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional