You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-17 - 21:59

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack2slot7s.osadl.org (updated Thu Sep 17, 2026 12:42:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
833517996052,4cyclictest1000-21mta-sts-daemon12:11:200
833518995343,6cyclictest835022-21systemd-run07:15:011
833517995033,10cyclictest12650irq/32-eth011:06:140
83351899491,38cyclictest946988-21ssh11:16:151
833518994736,8cyclictest1002-21mta-sts-daemon10:51:081
833518994736,8cyclictest1002-21mta-sts-daemon10:51:081
833518994635,8cyclictest927427-21cron10:45:001
833518994635,8cyclictest927427-21cron10:45:001
833519994536,5cyclictest913075-21fschecks_time10:21:052
833518994537,5cyclictest939630-21cut11:05:011
833519994436,5cyclictest959394-21mailstats11:36:212
833519994434,7cyclictest893998-21sh09:50:012
833517994434,7cyclictest47853-21cups-browsed10:53:150
833517994434,7cyclictest47853-21cups-browsed10:53:150
833517994431,10cyclictest12650irq/32-eth011:51:170
833517994414,23cyclictest943901-21sh11:11:140
833519994335,5cyclictest881790-21systemd-run09:30:012
833519994329,10cyclictest918963-21sh10:31:042
833518994332,7cyclictest47853-21cups-browsed09:46:191
833517994336,4cyclictest938180-21xargs11:01:230
833518994233,5cyclictest965450-21mailstats11:46:201
83351799422,31cyclictest910572-21ssh10:16:200
833519994134,4cyclictest871288-21unixbench-2d09:11:252
833519994133,5cyclictest951822-21systemd-run11:25:022
833518994132,6cyclictest1-21systemd10:00:021
833518994132,6cyclictest1-21systemd10:00:021
833517994128,7cyclictest0-21swapper/009:36:050
833519994031,6cyclictest47853-21cups-browsed10:46:272
833519994031,6cyclictest47853-21cups-browsed10:46:272
83351799402,29cyclictest971461-21ssh11:56:180
833519993932,4cyclictest933488-21idleruntime-cro10:55:002
833519993932,4cyclictest933488-21idleruntime-cro10:55:002
833519993931,5cyclictest992807-21mailstats12:31:212
83351899394,29cyclictest980513-21ssh12:11:201
833518993930,5cyclictest1002-21mta-sts-daemon12:26:161
83351799395,21cyclictest12650irq/32-eth009:31:110
833519993830,5cyclictest845408-21sed07:46:282
833517993830,5cyclictest876748-21missed_timers09:21:080
833517993825,10cyclictest265-21systemd-journal10:45:020
833517993825,10cyclictest265-21systemd-journal10:45:020
83351799382,31cyclictest983539-21sh12:16:190
833517993817,16cyclictest420-21avahi-daemon10:06:200
833519993729,5cyclictest878707-21cron09:25:012
833518993729,5cyclictest841522-21fschecks_time07:36:051
833518993728,5cyclictest968851-21unixbench_singl11:51:271
833518993727,6cyclictest853020-21systemd-run08:15:011
833517993731,4cyclictest936519-21idleruntime-cro11:00:010
833517993729,3cyclictest0-21swapper/010:36:100
833517993729,3cyclictest0-21swapper/010:36:100
833517993727,7cyclictest847146-21latency_hist07:55:020
833517993725,9cyclictest867888-21unixbench_multi09:01:250
833517993725,7cyclictest0-21swapper/011:21:150
833519993629,4cyclictest1003-21mta-sts-daemon11:42:272
833519993627,7cyclictest910514-21collect210:16:182
833519993626,6cyclictest265-21systemd-journal12:22:302
833519993625,8cyclictest420-21avahi-daemon09:37:422
833518993629,4cyclictest936527-21run-parts11:00:001
833518993628,5cyclictest875808-21latency_hist09:20:021
833518993627,5cyclictest848493-21idleruntime-cro08:00:011
833517993626,8cyclictest421-21dbus-daemon08:50:010
833517993626,7cyclictest890947-21latency_hist09:45:020
833517993626,3cyclictest0-21swapper/008:35:000
833517993626,3cyclictest0-21swapper/008:35:000
833517993623,8cyclictest12750irq/33-eth010:46:160
833517993623,8cyclictest12750irq/33-eth010:46:160
833517993623,3cyclictest0-21swapper/009:11:480
833519993527,5cyclictest979800-21aten_r2power_en12:11:042
833519993527,5cyclictest883503-21ssh09:31:232
833519993526,6cyclictest979181-21munin-plugin-st12:10:002
833519993526,6cyclictest979181-21munin-plugin-st12:10:002
833519993525,8cyclictest859810-21needreboot08:36:122
833519993525,8cyclictest859810-21needreboot08:36:122
833518993527,5cyclictest973919-21df12:01:041
833518993526,6cyclictest895404-21proc_pri09:51:221
833518993526,5cyclictest922278-21if_wlan010:36:101
833518993526,5cyclictest922278-21if_wlan010:36:101
833518993525,7cyclictest919916-21threads10:31:271
833518993525,6cyclictest869882-21sshd09:10:301
833518993523,9cyclictest928534-21missed_timers10:46:101
833518993523,9cyclictest928534-21missed_timers10:46:101
83351899352,26cyclictest940841-21ssh11:06:131
83351799354,24cyclictest959194-21ssh11:36:160
833517993527,5cyclictest900610-21/usr/sbin/munin10:01:240
833517993526,7cyclictest0-21swapper/008:06:110
833517993525,7cyclictest834379-21ntp_kernel_pll_07:11:120
833519993428,3cyclictest0-21swapper/209:20:002
833519993427,4cyclictest0-21swapper/211:31:202
833519993426,5cyclictest934334-21fschecks_count10:56:052
833519993426,5cyclictest865522-21fschecks_time08:56:052
83351999341,29cyclictest943998-21ssh11:11:182
833518993428,3cyclictest0-21swapper/112:35:011
833518993428,3cyclictest0-21swapper/107:20:001
833518993428,3cyclictest0-21swapper/107:20:001
833518993427,4cyclictest883194-21ps09:31:191
833518993424,7cyclictest910818-21timerandwakeup10:16:251
833517993428,4cyclictest860496-21munin-run08:40:000
833517993428,4cyclictest860496-21munin-run08:40:000
833517993428,3cyclictest868035-21kworker/u8:0-xprtiod09:16:250
833517993427,4cyclictest970073-21kworker/u8:11+xprtiod12:40:010
83351799342,23cyclictest916665-21ssh10:26:190
833519993325,5cyclictest939640-21systemd-run11:05:012
833519993325,5cyclictest865020-21systemd-run08:55:022
833519993324,5cyclictest849905-21unixbench_multi08:01:232
833518993327,3cyclictest0-21swapper/111:15:011
833518993327,3cyclictest0-21swapper/110:10:011
833518993324,7cyclictest913381-21needreboot10:21:101
83351899332,23cyclictest959140-21ssh11:36:141
83351899331,24cyclictest953145-21ssh11:26:181
83351899331,24cyclictest953145-21ssh11:26:181
83351799335,21cyclictest665-21snmpd10:24:080
833517993326,4cyclictest0-21swapper/011:46:250
833517993324,4cyclictest0-21swapper/012:01:030
833517993320,8cyclictest12750irq/33-eth009:51:150
833519993226,3cyclictest0-21swapper/209:45:012
833519993225,3cyclictest0-21swapper/207:36:112
833519993224,5cyclictest533-21NetworkManager10:40:432
833519993224,5cyclictest533-21NetworkManager10:40:432
833519993223,6cyclictest953456-21timerandwakeup11:26:252
833519993223,6cyclictest953456-21timerandwakeup11:26:252
833519993223,6cyclictest439-21systemd-logind07:25:012
833518993225,4cyclictest977008-21fschecks_time12:06:071
833518993225,4cyclictest977008-21fschecks_time12:06:071
833518993225,4cyclictest0-21swapper/112:20:011
833518993224,5cyclictest854418-21head08:16:251
833518993224,5cyclictest850033-21systemd-run08:05:011
833518993223,6cyclictest863515-21systemd-run08:50:011
83351899322,23cyclictest950056-21ssh11:21:161
833518993221,7cyclictest907873-21ls10:11:271
83351899321,23cyclictest889129-21ssh09:41:121
833517993225,3cyclictest0-21swapper/009:46:170
833517993223,6cyclictest869510-21cut09:10:020
833519993126,2cyclictest0-21swapper/210:10:012
833519993125,3cyclictest0-21swapper/207:31:212
833519993124,4cyclictest976176-21systemd-run12:05:022
833519993124,4cyclictest965813-21unixbench_singl11:46:242
833519993123,5cyclictest1001-21mta-sts-daemon12:35:142
833518993125,3cyclictest0-21swapper/111:45:011
833518993125,3cyclictest0-21swapper/109:26:161
833518993124,4cyclictest917115-21grep10:26:251
833518993124,4cyclictest901016-21if_wlan010:01:071
833517993125,4cyclictest850012-21munin-run08:05:010
83351799312,24cyclictest1-21systemd09:30:010
833517993120,9cyclictest840935-21unixbench_singl07:31:260
83351799311,8cyclictest986467-21ssh12:21:110
83351999307,13cyclictest1002-21mta-sts-daemon10:11:162
833519993024,3cyclictest0-21swapper/210:00:012
833519993024,3cyclictest0-21swapper/210:00:012
833519993023,4cyclictest835005-21sh07:15:002
833519993023,4cyclictest1002-21mta-sts-daemon11:58:352
833519993023,4cyclictest0-21swapper/207:20:002
833519993023,4cyclictest0-21swapper/207:20:002
833519993022,5cyclictest857500-21cron08:30:002
833519993022,5cyclictest853362-21aten_r2power_po08:16:032
83351999301,23cyclictest983445-21ssh12:16:152
833518993022,6cyclictest441-21thd08:30:021
833517993023,5cyclictest979171-21cron12:10:000
833517993023,5cyclictest979171-21cron12:10:000
833517993023,4cyclictest952812-21latency11:26:080
833517993023,4cyclictest952812-21latency11:26:080
833517993022,6cyclictest421-21dbus-daemon11:45:020
833517993021,6cyclictest906675-21/usr/sbin/munin10:11:100
833517993019,8cyclictest957864-21cron11:35:000
833517993014,10cyclictest151ktimers/011:16:180
83351999294,16cyclictest1000-21mta-sts-daemon10:26:122
833519992923,3cyclictest0-21swapper/211:06:152
833519992921,6cyclictest0-21swapper/209:05:012
833519992921,5cyclictest968850-21head11:51:272
833519992919,7cyclictest921865-21/usr/sbin/munin10:36:112
833519992919,7cyclictest921865-21/usr/sbin/munin10:36:112
833519992919,5cyclictest0-21swapper/208:41:162
833518992922,3cyclictest0-21swapper/109:21:491
833518992921,5cyclictest971871-21unixbench_singl11:56:231
833518992921,5cyclictest957853-21ssh11:35:001
833518992921,5cyclictest858025-21fschecks_count08:31:051
833518992921,5cyclictest858025-21fschecks_count08:31:051
833518992920,6cyclictest995988-21switchtime12:36:281
83351799299,14cyclictest1001-21mta-sts-daemon08:11:190
833517992924,3cyclictest0-21swapper/007:41:100
833517992919,4cyclictest12750irq/33-eth009:56:190
833517992919,4cyclictest12750irq/33-eth009:56:190
833517992918,8cyclictest836521-21sendmail07:20:000
833517992918,8cyclictest836521-21sendmail07:20:000
83351799290,4cyclictest171rcu_preempt07:46:170
833519992822,3cyclictest0-21swapper/209:51:542
833519992820,5cyclictest421-21dbus-daemon08:35:012
833519992820,5cyclictest421-21dbus-daemon08:35:012
833518992821,4cyclictest885937-21missed_timers09:36:101
833518992820,5cyclictest866526-21systemd-run09:00:011
833518992820,5cyclictest860490-21cron08:40:001
833518992820,5cyclictest860490-21cron08:40:001
833518992818,8cyclictest838010-21sh07:25:001
833517992823,2cyclictest0-21swapper/008:20:010
833517992822,3cyclictest842500-21awk07:40:010
83351799281,19cyclictest919603-21ssh10:31:170
83351999272,18cyclictest901436-21ntp_states10:01:192
833519992719,5cyclictest869519-21idleruntime-cro09:10:012
833519992719,4cyclictest0-21swapper/212:30:012
833519992718,6cyclictest0-21swapper/211:16:102
833518992719,6cyclictest844815-21needreboot07:46:101
833518992719,5cyclictest864938-21unixbench_singl08:51:261
833517992715,9cyclictest421-21dbus-daemon07:25:010
83351799271,19cyclictest989506-21ssh12:26:120
833519992620,3cyclictest0-21swapper/208:00:022
833518992620,3cyclictest841002-21munin-plugin-st07:35:011
833518992619,4cyclictest846988-21cron07:55:011
833518992615,8cyclictest867325-21needreboot09:01:101
833517992621,3cyclictest1002-21mta-sts-daemon12:31:320
833517992621,3cyclictest1000-21mta-sts-daemon08:56:160
833519992519,3cyclictest0-21swapper/207:41:122
833519992518,3cyclictest0-21swapper/208:10:012
833518992517,4cyclictest0-21swapper/112:21:111
833518992516,6cyclictest47853-21cups-browsed07:26:241
83351799255,4cyclictest171rcu_preempt07:26:170
833517992518,5cyclictest47853-21cups-browsed08:29:130
83351999242,16cyclictest855469-21ntp_states08:21:252
833519992418,3cyclictest0-21swapper/208:50:012
833519992416,5cyclictest852526-21wc08:11:202
833518992416,6cyclictest842775-21aten_r2power_cu07:41:001
833518992416,5cyclictest855032-21fschecks_count08:21:111
833517992417,4cyclictest427-21polkitd08:50:300
833517992412,9cyclictest0-21swapper/007:56:090
83351799241,18cyclictest421-21dbus-daemon08:45:010
833519992314,6cyclictest47853-21cups-browsed07:50:062
833518992318,2cyclictest0-21swapper/109:10:011
833518992310,3cyclictest171rcu_preempt08:06:021
833518992217,3cyclictest833516-21cyclictest08:41:121
83351999214,5cyclictest832177-21kworker/u8:1+xprtiod07:10:022
83351999214,5cyclictest832177-21kworker/u8:1+xprtiod07:10:022
833519992115,3cyclictest0-21swapper/207:30:012
83351799201,14cyclictest855361-21ntpq08:21:170
83352099192,8cyclictest0-21swapper/310:46:143
83352099192,8cyclictest0-21swapper/310:46:143
83351799181,6cyclictest833653-21latency_hist07:10:030
83351799181,6cyclictest833653-21latency_hist07:10:030
83352099174,6cyclictest0-21swapper/311:41:153
83352099174,5cyclictest0-21swapper/311:21:153
83352099172,9cyclictest0-21swapper/309:51:253
83352099171,8cyclictest0-21swapper/308:06:113
83352099171,10cyclictest0-21swapper/309:31:053
83352099165,8cyclictest0-21swapper/312:06:143
83352099165,8cyclictest0-21swapper/312:06:143
83352099163,7cyclictest0-21swapper/310:31:033
83352099163,7cyclictest0-21swapper/309:16:083
83352099163,5cyclictest0-21swapper/307:36:113
83352099162,8cyclictest0-21swapper/311:11:143
83352099162,8cyclictest0-21swapper/309:21:013
83352099161,9cyclictest0-21swapper/311:01:123
83352099161,9cyclictest0-21swapper/309:36:143
83352099161,8cyclictest0-21swapper/310:11:193
83352099161,7cyclictest0-21swapper/308:46:163
83352099161,6cyclictest0-21swapper/307:11:123
833518991610,3cyclictest0-21swapper/107:10:031
833518991610,3cyclictest0-21swapper/107:10:031
83352099153,7cyclictest0-21swapper/309:56:173
83352099153,7cyclictest0-21swapper/309:56:173
83352099153,7cyclictest0-21swapper/309:11:173
83352099153,4cyclictest0-21swapper/308:01:133
83352099152,7cyclictest0-21swapper/310:21:113
83352099152,6cyclictest0-21swapper/309:01:173
83352099151,8cyclictest0-21swapper/312:31:153
83352099151,8cyclictest0-21swapper/310:56:123
83352099151,6cyclictest0-21swapper/310:40:433
83352099151,6cyclictest0-21swapper/310:40:433
83352099151,6cyclictest0-21swapper/308:31:133
83352099151,6cyclictest0-21swapper/308:31:133
83352099151,6cyclictest0-21swapper/307:51:113
83352099151,6cyclictest0-21swapper/307:41:093
83352099144,7cyclictest0-21swapper/311:46:113
83352099142,8cyclictest0-21swapper/311:06:133
83352099142,7cyclictest0-21swapper/312:21:133
83352099142,7cyclictest0-21swapper/312:11:243
83352099142,7cyclictest0-21swapper/311:56:103
83352099142,7cyclictest0-21swapper/310:51:143
83352099142,7cyclictest0-21swapper/310:51:143
83352099142,7cyclictest0-21swapper/310:36:093
83352099142,7cyclictest0-21swapper/310:36:093
83352099142,7cyclictest0-21swapper/310:16:143
83352099142,6cyclictest0-21swapper/311:51:143
83352099142,6cyclictest0-21swapper/309:50:033
83352099141,8cyclictest0-21swapper/312:36:183
83352099141,8cyclictest0-21swapper/311:36:123
83352099141,8cyclictest0-21swapper/311:26:193
83352099141,8cyclictest0-21swapper/311:26:193
83352099141,7cyclictest0-21swapper/312:15:303
83352099141,7cyclictest0-21swapper/311:16:023
83352099141,6cyclictest0-21swapper/308:41:143
83352099141,6cyclictest0-21swapper/308:21:223
83352099141,6cyclictest0-21swapper/308:16:143
83352099141,5cyclictest0-21swapper/308:51:143
83352099133,6cyclictest0-21swapper/309:10:013
83352099132,6cyclictest0-21swapper/312:01:033
83352099132,6cyclictest0-21swapper/307:35:023
83352099131,9cyclictest0-21swapper/311:31:023
83352099131,7cyclictest0-21swapper/310:06:173
83352099131,7cyclictest0-21swapper/309:26:123
83352099131,7cyclictest0-21swapper/308:36:173
83352099131,7cyclictest0-21swapper/308:36:173
83352099131,6cyclictest0-21swapper/307:56:163
83352099131,6cyclictest0-21swapper/307:46:153
83352099122,6cyclictest0-21swapper/310:01:023
83352099121,8cyclictest0-21swapper/309:41:063
83352099121,7cyclictest0-21swapper/312:26:113
83352099121,6cyclictest0-21swapper/308:56:133
83352099121,6cyclictest0-21swapper/308:26:133
83352099121,6cyclictest0-21swapper/308:11:123
83352099120,9cyclictest431ktimers/307:10:013
83352099120,9cyclictest431ktimers/307:10:013
83352099114,5cyclictest0-21swapper/307:25:013
83352099112,6cyclictest0-21swapper/310:26:103
8335209991,5cyclictest0-21swapper/307:25:303
8335209991,5cyclictest0-21swapper/307:16:133
8335209991,5cyclictest0-21swapper/307:16:133
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional