You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-21 - 14:42

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack2slot7s.osadl.org (updated Fri Aug 21, 2026 12:42:58)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
4193583995346,3cyclictest0-21swapper/212:00:592
4193583995346,3cyclictest0-21swapper/212:00:592
4193583995140,8cyclictest114852-21munin-run11:20:012
4193581994839,6cyclictest151ktimers/009:46:030
419358399473,38cyclictest146512-21ssh12:11:012
4193581994637,4cyclictest0-21swapper/009:56:100
4193582994535,5cyclictest1000-21mta-sts-daemon10:06:021
4193583994333,7cyclictest38668-21idleruntime-cro09:15:012
419358399431,33cyclictest58154-21ssh09:46:022
4193582994234,5cyclictest41742-21sendmail09:20:011
4193582994232,7cyclictest154439-21/usr/sbin/munin12:25:011
4193581994227,9cyclictest13955-21ntpq07:55:590
4193582994134,4cyclictest0-21swapper/110:36:041
4193581994134,4cyclictest102682-21sendmail11:00:000
4193581994134,4cyclictest102682-21sendmail11:00:000
4193581994124,14cyclictest27145-21fschecks_time08:40:520
419358199412,32cyclictest52110-21ssh09:35:580
419358199412,32cyclictest52110-21ssh09:35:580
4193581994114,9cyclictest48226-21sh09:30:230
4193581994114,9cyclictest48226-21sh09:30:230
4193582994032,5cyclictest115525-21aten_r2power_vo11:20:501
4193581994034,4cyclictest163573-21munin-run12:40:010
4193581994023,13cyclictest8016-21ntpq07:36:040
4193581994023,12cyclictest148655-21kworker/u8:11+rpciod12:30:570
4193581994013,11cyclictest6990-21sed07:31:100
419358399398,22cyclictest82742-21ssh10:26:072
4193583993933,3cyclictest0-21swapper/208:20:002
4193582993930,6cyclictest421-21dbus-daemon12:15:021
4193582993926,10cyclictest107254-21switchtime11:06:081
4193581993935,2cyclictest0-21swapper/011:25:540
4193581993916,8cyclictest17024-21ntpq08:06:000
4193581993916,8cyclictest17024-21ntpq08:06:000
4193581993914,9cyclictest115417-21hostname11:20:490
4193581993913,11cyclictest98844-21ssh10:52:340
4193581993913,11cyclictest98844-21ssh10:52:340
4193581993913,11cyclictest69633-21sh10:05:410
4193581993912,11cyclictest157013-21ssh12:28:340
4193583993830,5cyclictest27138-21fschecks_time08:40:522
4193583993829,4cyclictest0-21swapper/211:51:132
4193583993827,7cyclictest3397920-21cups-browsed12:21:562
4193582993831,4cyclictest163594-21sendmail12:40:011
4193582993830,5cyclictest87459-21munin-plugin-st10:35:011
4193582993830,5cyclictest50947-21cron09:35:011
4193582993830,5cyclictest50947-21cron09:35:011
4193582993829,6cyclictest2621-21sendmail07:20:001
4193582993829,6cyclictest2621-21sendmail07:20:001
4193582993829,5cyclictest0-21swapper/111:40:161
4193582993827,9cyclictest97689-21needreboot10:50:561
4193582993827,9cyclictest97689-21needreboot10:50:561
4193581993831,4cyclictest130105-21ssh11:45:010
4193581993825,8cyclictest493-21ntpq07:11:020
4193581993825,8cyclictest493-21ntpq07:11:020
4193581993825,8cyclictest34966-21ntpq09:05:570
4193581993825,8cyclictest30540-21ntpq08:51:030
4193581993825,8cyclictest19993-21ntpq08:16:000
4193581993824,9cyclictest33586-21ntpq09:01:040
4193581993824,8cyclictest110078-21ntpq11:11:070
4193581993814,8cyclictest47189-21sh09:27:530
4193581993814,7cyclictest35603-21diskmemload11:33:480
4193581993813,7cyclictest623-21sshd09:45:010
4193581993813,7cyclictest623-21sshd09:45:010
4193581993813,7cyclictest35603-21diskmemload09:12:270
4193581993812,8cyclictest35603-21diskmemload12:20:360
4193581993811,8cyclictest10968-21ntpq07:45:590
4193581993810,7cyclictest151381-21munin-run12:20:010
4193583993731,3cyclictest0-21swapper/211:15:022
4193583993729,5cyclictest128162-21sh11:40:572
4193581993723,8cyclictest140467-21ntpq12:01:020
4193581993723,8cyclictest140467-21ntpq12:01:020
4193581993722,12cyclictest76515-21ntpq10:16:010
4193581993721,3cyclictest82541-21ntpq10:25:590
4193581993717,14cyclictest0-21swapper/010:01:050
4193581993716,7cyclictest35603-21diskmemload12:12:490
4193581993716,7cyclictest114408-21ssh11:18:370
4193581993716,16cyclictest3397920-21cups-browsed09:54:130
4193581993716,16cyclictest3397920-21cups-browsed09:54:130
4193581993714,8cyclictest135806-21ssh11:53:550
4193581993714,7cyclictest35603-21diskmemload09:16:240
4193581993714,7cyclictest130464-21sh11:45:240
4193581993714,7cyclictest108249-21ssh11:08:200
4193581993714,17cyclictest0-21swapper/008:10:570
4193581993714,16cyclictest0-21swapper/007:21:040
4193581993713,8cyclictest93149-21ssh10:43:450
4193581993712,9cyclictest142943-21awk12:05:530
4193581993712,10cyclictest44834-21awk09:25:010
4193581993710,8cyclictest81388-21munin-run10:25:010
4193581993710,8cyclictest24563-21ntpq08:31:020
419358399368,5cyclictest171rcu_preempt10:46:032
419358399368,5cyclictest171rcu_preempt10:46:032
4193583993627,6cyclictest1002-21mta-sts-daemon10:54:282
4193583993627,6cyclictest1002-21mta-sts-daemon10:54:282
4193583993627,5cyclictest104037-21proc_pri11:01:042
419358399360,5cyclictest36-21ksoftirqd/210:40:592
4193582993630,3cyclictest0-21swapper/109:05:011
4193582993628,5cyclictest31092-21munin-plugin-st08:55:011
4193582993627,6cyclictest55828-21unixbench_singl09:41:171
4193582993627,6cyclictest55828-21unixbench_singl09:41:171
4193582993627,5cyclictest421-21dbus-daemon10:03:451
4193581993627,4cyclictest85640-21ntpq10:31:010
4193581993624,8cyclictest21458-21ntpq08:20:560
4193581993624,8cyclictest2079-21ntpq07:16:070
4193581993624,8cyclictest2079-21ntpq07:16:070
4193581993624,7cyclictest32000-21ntpq08:56:010
4193581993621,11cyclictest5025-21ntpq07:25:590
4193581993614,7cyclictest35603-21diskmemload10:47:230
4193581993614,7cyclictest35603-21diskmemload10:47:230
4193581993613,13cyclictest104544-21ssh11:01:140
4193581993612,9cyclictest138602-21ssh11:58:070
4193583993529,3cyclictest0-21swapper/211:46:122
4193583993528,4cyclictest117909-21cron11:25:012
4193583993527,5cyclictest1000-21mta-sts-daemon09:39:292
4193583993527,5cyclictest1000-21mta-sts-daemon09:39:292
4193583993525,3cyclictest171rcu_preempt08:01:032
4193583993519,6cyclictest421-21dbus-daemon10:08:432
4193582993528,4cyclictest1003-21mta-sts-daemon10:26:031
4193582993527,5cyclictest20639-21systemd-run08:20:011
4193582993526,6cyclictest101102-21mailstats10:56:091
4193582993526,6cyclictest101102-21mailstats10:56:091
4193582993526,4cyclictest145457-21latency_hist12:10:041
4193582993519,4cyclictest171rcu_preempt12:31:011
419358199359,8cyclictest126482-21ssh11:38:150
4193581993528,4cyclictest0-21swapper/010:35:530
4193581993524,7cyclictest15467-21ntpq08:00:580
4193581993523,7cyclictest25988-21ntpq08:36:000
4193581993522,8cyclictest28993-21ntpq08:45:580
4193581993514,15cyclictest0-21swapper/008:25:590
4193581993513,16cyclictest0-21swapper/010:10:570
419358399343,23cyclictest566-21gmain08:05:592
419358399343,23cyclictest566-21gmain08:05:592
4193583993429,2cyclictest0-21swapper/208:00:012
4193583993425,6cyclictest31097-21munin-plugin-st08:55:012
4193583993422,9cyclictest85028-21aten_r2power_po10:30:492
4193582993428,3cyclictest0-21swapper/112:01:071
4193582993428,3cyclictest0-21swapper/112:01:071
4193582993425,6cyclictest11527-21unixbench_singl07:46:111
4193582993423,7cyclictest420-21avahi-daemon10:15:531
4193581993422,8cyclictest9502-21ntpq07:41:000
4193581993422,8cyclictest12462-21ntpq07:50:570
419358399338,6cyclictest351ktimers/211:05:572
4193583993327,3cyclictest0-21swapper/210:21:022
4193583993326,4cyclictest32561-21kworker/u8:8-xprtiod09:40:512
4193583993326,4cyclictest32561-21kworker/u8:8-xprtiod09:40:512
4193583993326,4cyclictest0-21swapper/212:25:522
4193583993325,5cyclictest101404-21head10:56:122
4193583993325,5cyclictest101404-21head10:56:122
4193583993325,3cyclictest0-21swapper/211:36:092
4193583993323,3cyclictest171rcu_preempt07:16:062
4193583993323,3cyclictest171rcu_preempt07:16:062
4193583993317,8cyclictest171rcu_preempt08:11:002
419358299336,19cyclictest428-21mta-sts-daemon08:20:561
419358299333,24cyclictest46190-21ssh09:26:071
4193582993326,3cyclictest0-21swapper/109:10:011
4193582993325,5cyclictest61212-21ntp_kernel_pll_09:50:591
4193582993325,5cyclictest61212-21ntp_kernel_pll_09:50:591
4193582993324,6cyclictest42378-21aten_r2power_po09:20:501
4193582993324,5cyclictest95029-21mailstats10:46:091
4193582993324,5cyclictest95029-21mailstats10:46:091
419358299331,27cyclictest112942-21rm11:15:571
4193583993226,3cyclictest4192191-21kworker/u8:7-xprtiod08:30:022
4193583993226,3cyclictest0-21swapper/209:00:012
4193583993226,3cyclictest0-21swapper/208:35:022
4193583993224,5cyclictest88025-21aten_r2power_fr10:35:492
4193583993216,3cyclictest171rcu_preempt09:56:012
4193583993214,14cyclictest420-21avahi-daemon09:21:012
419358299324,24cyclictest122183-21ssh11:31:001
4193582993227,2cyclictest0-21swapper/108:45:581
4193582993223,7cyclictest9465-21ntp_kernel_err07:40:581
4193583993123,5cyclictest485-21in:imuxsock12:00:012
4193583993123,5cyclictest158614-21sh12:30:572
4193583993123,5cyclictest10058-21head07:41:112
4193583993122,5cyclictest351ktimers/212:16:032
4193583993122,5cyclictest351ktimers/207:51:002
4193583993119,3cyclictest171rcu_preempt10:10:572
4193583993117,3cyclictest171rcu_preempt09:31:072
4193583993117,3cyclictest171rcu_preempt09:31:072
4193583993117,2cyclictest171rcu_preempt09:25:322
4193583993115,6cyclictest171rcu_preempt10:16:002
4193583993115,3cyclictest171rcu_preempt11:34:172
4193582993124,4cyclictest1001-21mta-sts-daemon11:48:311
4193582993123,4cyclictest0-21swapper/111:55:581
4193582993122,6cyclictest3397920-21cups-browsed08:35:511
4193582993120,8cyclictest58286-21ntp_offset09:46:071
4193583993021,5cyclictest351ktimers/212:06:272
4193583993021,3cyclictest171rcu_preempt08:36:012
4193583993016,3cyclictest171rcu_preempt12:35:542
419358399301,23cyclictest119182-21ssh11:26:022
419358299307,13cyclictest1000-21mta-sts-daemon08:56:061
4193582993024,3cyclictest0-21swapper/112:16:041
4193582993024,3cyclictest0-21swapper/110:10:531
4193582993024,3cyclictest0-21swapper/107:36:011
4193582993023,4cyclictest0-21swapper/109:36:031
4193582993023,4cyclictest0-21swapper/109:36:031
4193582993022,3cyclictest0-21swapper/111:35:301
4193582993021,7cyclictest37396-21sed09:11:091
4193582993021,5cyclictest103230-21aten_r2power_en11:00:471
4193582993020,6cyclictest420-21avahi-daemon07:10:011
4193582993014,13cyclictest271ktimers/108:41:011
419358399295,9cyclictest36-21ksoftirqd/209:05:592
419358399294,9cyclictest36-21ksoftirqd/208:46:022
419358399292,3cyclictest61324-21ssh09:51:042
419358399292,3cyclictest61324-21ssh09:51:042
4193583992922,3cyclictest0-21swapper/209:16:012
4193583992920,6cyclictest4660-21grep07:25:512
4193583992917,6cyclictest171rcu_preempt07:11:032
4193583992917,6cyclictest171rcu_preempt07:11:032
4193583992916,3cyclictest171rcu_preempt07:30:562
4193583992914,3cyclictest171rcu_preempt08:20:562
4193583992910,3cyclictest171rcu_preempt10:01:082
419358299294,16cyclictest958-21sendmail-mta07:50:591
4193582992922,4cyclictest134859-21tail11:51:131
4193582992921,5cyclictest64056-21latency09:55:531
419358299291,23cyclictest119079-21ssh11:25:581
419358399286,9cyclictest351ktimers/207:36:152
4193583992812,3cyclictest171rcu_preempt07:45:502
4193582992820,5cyclictest694-21cron07:25:001
4193582992820,5cyclictest3397920-21cups-browsed10:44:181
4193582992817,8cyclictest6476-21ntp_kernel_pll_07:30:581
419358299281,21cyclictest79602-21ssh10:21:021
4193583992718,6cyclictest3644-21proc_pri07:21:052
4193583992714,2cyclictest171rcu_preempt09:01:052
4193583992712,3cyclictest171rcu_preempt07:10:022
4193582992722,2cyclictest0-21swapper/111:11:021
4193582992721,3cyclictest0-21swapper/112:25:481
4193582992720,4cyclictest13732-21kernelversion07:55:551
4193582992720,4cyclictest0-21swapper/108:35:011
4193582992718,6cyclictest18831-21timerandwakeup08:11:091
4193581992717,7cyclictest4193539-21munin-run07:10:010
4193582992619,4cyclictest16748-21kernelversion08:05:541
4193582992619,4cyclictest16748-21kernelversion08:05:541
4193582992618,6cyclictest23205-21sendmail_mailtr08:26:061
4193582992618,6cyclictest0-21swapper/107:10:481
4193582992618,6cyclictest0-21swapper/107:10:481
4193582992213,6cyclictest15664-21ps08:01:051
419358299221,16cyclictest5135-21ps07:26:041
419358499184,7cyclictest0-21swapper/311:30:593
419358499183,7cyclictest0-21swapper/310:46:043
419358499183,7cyclictest0-21swapper/310:46:043
419358499182,8cyclictest0-21swapper/312:35:593
419358499182,8cyclictest0-21swapper/311:46:053
419358499182,8cyclictest0-21swapper/309:31:033
419358499182,8cyclictest0-21swapper/309:31:033
419358499173,7cyclictest0-21swapper/310:56:043
419358499173,7cyclictest0-21swapper/310:56:043
419358499172,9cyclictest0-21swapper/309:41:063
419358499172,9cyclictest0-21swapper/309:41:063
419358499172,8cyclictest0-21swapper/310:30:573
419358499172,7cyclictest0-21swapper/308:45:573
419358499172,10cyclictest0-21swapper/309:36:013
419358499172,10cyclictest0-21swapper/309:36:013
419358499171,8cyclictest0-21swapper/310:21:043
419358499171,7cyclictest0-21swapper/307:36:013
419358499165,7cyclictest0-21swapper/309:26:043
419358499163,9cyclictest0-21swapper/309:21:193
419358499163,6cyclictest0-21swapper/311:11:033
419358499162,8cyclictest0-21swapper/310:50:593
419358499162,8cyclictest0-21swapper/310:50:593
419358499162,8cyclictest0-21swapper/310:16:023
419358499161,8cyclictest0-21swapper/311:00:573
419358499153,7cyclictest0-21swapper/310:25:583
419358499152,9cyclictest0-21swapper/310:40:023
419358499152,8cyclictest0-21swapper/312:23:483
419358499152,8cyclictest0-21swapper/312:00:013
419358499152,8cyclictest0-21swapper/309:16:003
419358499152,7cyclictest0-21swapper/312:00:533
419358499152,7cyclictest0-21swapper/312:00:533
419358499152,7cyclictest0-21swapper/311:35:593
419358499152,7cyclictest0-21swapper/310:05:563
419358499152,6cyclictest0-21swapper/309:00:583
419358499151,8cyclictest0-21swapper/311:25:583
419358499151,8cyclictest0-21swapper/311:21:033
419358499151,8cyclictest0-21swapper/310:40:583
419358499151,8cyclictest0-21swapper/310:10:583
419358499151,7cyclictest0-21swapper/309:55:573
419358499151,7cyclictest0-21swapper/308:00:583
419358499150,7cyclictest0-21swapper/307:45:593
419358499143,7cyclictest0-21swapper/311:15:543
419358499143,7cyclictest0-21swapper/311:05:573
419358499143,6cyclictest0-21swapper/312:30:573
419358499143,6cyclictest0-21swapper/307:35:023
419358499142,8cyclictest0-21swapper/309:50:493
419358499142,8cyclictest0-21swapper/309:50:493
419358499142,7cyclictest0-21swapper/312:25:513
419358499141,8cyclictest0-21swapper/307:45:013
419358499141,7cyclictest0-21swapper/309:06:003
419358499141,7cyclictest0-21swapper/308:55:013
419358499141,7cyclictest0-21swapper/308:15:003
419358499141,7cyclictest0-21swapper/307:50:573
419358499141,6cyclictest0-21swapper/308:41:003
419358499141,6cyclictest0-21swapper/307:16:013
419358499141,6cyclictest0-21swapper/307:16:013
419358499133,7cyclictest0-21swapper/308:56:033
419358499132,7cyclictest0-21swapper/312:05:583
419358499132,6cyclictest0-21swapper/308:40:023
419358499132,5cyclictest0-21swapper/309:45:133
419358499131,8cyclictest0-21swapper/311:50:563
419358499131,7cyclictest0-21swapper/311:40:583
419358499131,7cyclictest0-21swapper/310:05:023
419358499131,7cyclictest0-21swapper/309:10:573
419358499131,7cyclictest0-21swapper/307:25:583
419358499131,6cyclictest0-21swapper/307:21:013
419358499131,5cyclictest0-21swapper/308:30:573
419358499131,5cyclictest0-21swapper/308:05:573
419358499131,5cyclictest0-21swapper/308:05:573
419358499122,7cyclictest0-21swapper/312:10:503
419358499122,6cyclictest0-21swapper/308:25:573
419358499122,5cyclictest0-21swapper/307:11:003
419358499122,5cyclictest0-21swapper/307:11:003
419358499121,7cyclictest0-21swapper/308:20:003
419358499121,6cyclictest0-21swapper/307:56:013
419358499113,6cyclictest0-21swapper/312:15:483
419358499112,5cyclictest0-21swapper/308:20:483
41935849991,6cyclictest0-21swapper/307:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional