You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-19 - 12:00

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Sat Sep 19, 2026 00:42:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3594789995439,11cyclictest3698522-21fake-hwclock23:01:001
3594789995040,6cyclictest3750794-21gltestperf00:26:081
3594788995042,5cyclictest1-21systemd19:30:010
3594788995042,5cyclictest1-21systemd19:30:010
3594788994715,13cyclictest3674818-21ntpq22:21:120
3594789994639,4cyclictest0-21swapper/119:36:151
3594789994614,3cyclictest171rcu_preempt00:21:191
3594790994537,5cyclictest3602293-21systemd-run19:35:012
3594789994539,3cyclictest0-21swapper/123:41:251
3594788994537,5cyclictest421-21dbus-daemon21:30:010
3594788994537,5cyclictest3695791-21fw_packets22:56:070
3594788994518,9cyclictest3625711-21ntpq20:51:150
3594790994413,7cyclictest171rcu_preempt00:06:152
3594789994434,6cyclictest3663251-21unixbench_singl22:01:261
3594789994434,6cyclictest3663251-21unixbench_singl22:01:261
3594789994433,7cyclictest3746861-21sendmail00:20:011
359478999443,32cyclictest3736054-21ssh00:01:201
3594789994334,6cyclictest3697960-21grep23:00:001
3594788994323,14cyclictest3603142-21ntpq19:36:150
3594788994316,9cyclictest3643379-21sh21:30:260
3594788994316,11cyclictest3682182-21ssh22:33:220
3594788994315,8cyclictest3683987-21ntpq22:36:160
3594788994313,12cyclictest3702797-21sed23:06:250
3594789994232,6cyclictest1001-21mta-sts-daemon19:16:131
3594789994231,8cyclictest421-21dbus-daemon21:10:011
3594788994228,11cyclictest3609801-21systemd-run20:00:020
3594788994224,12cyclictest3613622-21ntpq20:11:130
3594788994217,9cyclictest3758440-21sh00:38:060
3594788994217,9cyclictest3758440-21sh00:38:060
3594790994133,5cyclictest3710183-21sendmail23:20:012
3594790994133,5cyclictest3681594-21grep22:31:342
3594790994131,6cyclictest3639986-21ssh21:25:022
3594790994129,9cyclictest3599301-21sh19:25:012
3594789994134,3cyclictest0-21swapper/123:15:011
3594789994132,6cyclictest1-21systemd22:55:011
3594789994132,6cyclictest1-21systemd22:55:011
3594789994131,6cyclictest3719325-21munin-run23:35:011
3594789994129,9cyclictest3676666-21ssh22:25:021
3594789994112,2cyclictest0-21swapper/121:36:201
3594788994135,4cyclictest12650irq/32-eth023:56:130
3594788994130,8cyclictest3694929-21true22:55:010
3594788994130,8cyclictest3694929-21true22:55:010
3594788994117,7cyclictest3653585-21ntpq21:46:190
3594788994117,7cyclictest3623799-21/usr/sbin/munin20:46:070
3594788994114,8cyclictest3677952-21ntpq22:26:160
3594788994113,9cyclictest3637451-21/usr/sbin/munin21:21:020
3594790994031,5cyclictest3690241-21proc_pri22:46:192
3594790994031,5cyclictest3690241-21proc_pri22:46:192
3594790994029,6cyclictest0-21swapper/200:16:152
3594790994028,9cyclictest3650705-21mailstats21:41:222
3594789994031,6cyclictest421-21dbus-daemon00:00:011
3594788994031,3cyclictest0-21swapper/000:21:130
3594788994016,16cyclictest0-21swapper/020:36:150
3594788994015,9cyclictest3709884-21sh23:19:110
3594788994015,9cyclictest3705780-21timerwakeupswit23:11:250
3594790993932,4cyclictest1-21systemd23:15:012
3594790993931,5cyclictest3731555-21systemd-run23:55:012
3594790993927,9cyclictest3641058-21missed_timers21:26:082
3594790993926,10cyclictest3711679-21mailstats23:21:222
359479099391,33cyclictest3757268-21ssh00:36:162
359479099391,33cyclictest3757268-21ssh00:36:162
3594788993933,3cyclictest0-21swapper/021:51:190
3594788993931,5cyclictest427-21polkitd22:14:560
3594788993926,10cyclictest1896942-21cups-browsed21:11:150
3594788993925,9cyclictest3627233-21ntpq20:56:190
3594788993925,8cyclictest3628644-21ntpq21:01:110
3594788993925,8cyclictest3598679-21ntpq19:21:130
3594788993921,12cyclictest3742074-21ntpq00:11:170
3594788993915,7cyclictest3649139-21diskmemload21:40:000
3594788993913,9cyclictest3712892-21sh23:23:590
3594790993831,4cyclictest3594787-21cyclictest22:21:142
3594789993830,5cyclictest3634712-21tr21:16:111
3594789993829,5cyclictest0-21swapper/120:56:141
3594789993813,8cyclictest0-21swapper/122:36:221
3594788993826,7cyclictest3606123-21ntpq19:46:110
3594788993825,8cyclictest3607678-21ntpq19:51:130
3594788993823,12cyclictest151ktimers/023:37:260
3594788993816,8cyclictest3731254-21diskmemload23:54:090
3594788993815,7cyclictest3664220-21ssh22:04:250
3594788993815,7cyclictest3664220-21ssh22:04:250
3594788993814,8cyclictest3716734-21rm23:30:460
3594788993812,8cyclictest750-21ntpd19:11:130
3594788993812,8cyclictest3612158-21ntpq20:06:130
359479099373,25cyclictest3702359-21ssh23:06:172
3594790993731,3cyclictest0-21swapper/222:20:012
3594790993729,5cyclictest3603776-21systemctl19:40:002
3594790993729,4cyclictest0-21swapper/221:56:222
3594790993728,5cyclictest3615807-21systemd-run20:20:012
3594790993727,6cyclictest3623299-21systemd-run20:45:012
3594789993730,4cyclictest1003-21mta-sts-daemon22:28:181
3594789993729,5cyclictest421-21dbus-daemon23:20:011
3594789993728,6cyclictest1000-21mta-sts-daemon21:14:001
3594789993728,5cyclictest3644282-21ntp_kernel_pll_21:31:131
3594788993730,3cyclictest0-21swapper/000:16:050
3594788993727,7cyclictest3611280-21munin-plugin-st20:05:020
3594788993724,8cyclictest3618143-21ntpq20:26:120
3594788993724,7cyclictest3726737-21ntpq23:46:160
3594788993722,12cyclictest3597167-21ntpq19:16:140
3594788993717,8cyclictest3630769-21diskmemload23:00:050
3594788993714,8cyclictest3630769-21diskmemload21:58:080
3594788993711,7cyclictest3630769-21diskmemload00:34:370
359479099368,21cyclictest3751215-21ssh00:26:172
3594790993628,5cyclictest3699835-21unixbench_singl23:01:272
3594789993628,5cyclictest3720386-21latency23:36:111
3594788993628,5cyclictest265-21systemd-journal22:45:010
3594788993626,7cyclictest3735230-21aten_r2power_cu00:01:020
3594788993625,8cyclictest3691866-21sh22:50:010
3594788993625,8cyclictest3691866-21sh22:50:010
3594788993625,7cyclictest3616678-21ntpq20:21:130
3594788993624,8cyclictest3601719-21ntpq19:31:150
3594788993624,7cyclictest3630220-21ntpq21:06:160
3594788993624,7cyclictest3622681-21ntpq20:41:140
359478899362,24cyclictest3665701-21ssh22:06:160
359478899362,24cyclictest3665701-21ssh22:06:160
3594788993622,11cyclictest3619691-21ntpq20:31:180
3594788993617,15cyclictest3635433-21ssh21:16:420
3594788993615,18cyclictest0-21swapper/019:41:130
3594788993612,7cyclictest3738235-21/usr/sbin/munin00:06:080
3594788993612,7cyclictest3652075-21ssh21:44:340
3594790993528,4cyclictest3719315-21rm23:35:012
3594790993528,4cyclictest3596262-21cron19:15:012
3594790993527,5cyclictest1000-21mta-sts-daemon22:54:032
3594790993527,5cyclictest1000-21mta-sts-daemon22:54:032
3594790993527,3cyclictest0-21swapper/222:40:012
3594790993526,6cyclictest421-21dbus-daemon00:11:172
359479099351,26cyclictest3748071-21ssh00:21:122
3594789993527,5cyclictest421-21dbus-daemon21:56:351
3594789993527,4cyclictest0-21swapper/120:36:141
3594789993526,7cyclictest421-21dbus-daemon20:05:011
3594789993524,8cyclictest3682708-21cron22:35:011
3594788993524,7cyclictest3751245-21ntpq00:26:180
3594788993523,7cyclictest3714435-21ntpq23:26:120
3594788993522,8cyclictest3615148-21ntpq20:16:130
3594788993519,11cyclictest3722730-21ssh23:40:140
359479099345,15cyclictest3696222-21ssh22:56:162
3594790993427,3cyclictest0-21swapper/223:40:012
3594790993426,5cyclictest3734598-21munin-plugin-st00:00:002
359479099342,25cyclictest1896939-21cupsd21:31:172
359478999346,15cyclictest428-21mta-sts-daemon21:21:181
3594789993429,2cyclictest0-21swapper/119:35:011
3594789993427,4cyclictest3656246-21kernelversion21:51:081
359478999342,24cyclictest3665644-21ssh22:06:141
359478999342,24cyclictest3665644-21ssh22:06:141
3594789993419,3cyclictest171rcu_preempt00:33:561
359478999341,27cyclictest3649560-21ssh21:40:321
3594788993422,8cyclictest3671736-21ntpq22:16:130
3594790993325,4cyclictest3665655-21kthreadd22:06:142
3594790993325,4cyclictest3665655-21kthreadd22:06:142
3594790993324,5cyclictest3632557-21sed21:11:302
3594790993318,11cyclictest351ktimers/219:41:152
359478999339,3cyclictest0-21swapper/120:41:121
3594789993326,4cyclictest1002-21mta-sts-daemon23:48:231
3594789993326,3cyclictest0-21swapper/122:20:021
3594789993326,3cyclictest0-21swapper/121:46:091
3594789993326,3cyclictest0-21swapper/100:06:171
3594789993325,5cyclictest3742151-21proc_pri00:11:201
3594789993324,5cyclictest0-21swapper/120:51:171
3594789993315,2cyclictest0-21swapper/123:51:161
3594790993224,5cyclictest3647540-21proc_pri21:36:232
3594790993224,5cyclictest0-21swapper/220:00:012
3594789993224,5cyclictest3605270-21cron19:45:011
3594789993224,4cyclictest1003-21mta-sts-daemon21:26:511
3594789993220,2cyclictest171rcu_preempt20:26:131
3594790993125,3cyclictest0-21swapper/220:40:012
3594790993124,4cyclictest3725478-21systemd-run23:45:022
3594790993123,5cyclictest3737711-21systemd-run00:05:012
3594790993122,6cyclictest0-21swapper/221:10:022
359479099312,23cyclictest3635194-21ssh21:16:232
3594789993124,4cyclictest3691894-21systemd-run22:50:021
3594789993124,4cyclictest3691894-21systemd-run22:50:021
3594789993116,2cyclictest171rcu_preempt23:27:061
3594789993112,2cyclictest171rcu_preempt22:41:161
359478999311,13cyclictest3702221-21ntpq23:06:121
3594790993023,4cyclictest3597212-21ntp_offset19:16:152
3594790993023,4cyclictest0-21swapper/223:46:262
3594790993022,6cyclictest0-21swapper/221:55:022
3594790993020,8cyclictest3618093-21needreboot20:26:102
359478999302,21cyclictest3668810-21ssh22:11:171
3594789993019,3cyclictest171rcu_preempt19:26:161
3594789993019,3cyclictest171rcu_preempt19:26:161
3594789993019,3cyclictest171rcu_preempt19:21:131
3594789993019,3cyclictest171rcu_preempt00:36:211
3594789993019,3cyclictest171rcu_preempt00:36:211
3594789993017,3cyclictest171rcu_preempt20:31:201
359479099295,16cyclictest428-21mta-sts-daemon22:26:132
3594790992922,4cyclictest3627790-21kthreadd21:00:012
3594790992922,4cyclictest3613038-21aten_r2power_cu20:11:032
3594790992921,6cyclictest3670517-21awk22:15:002
3594790992921,5cyclictest3607310-21gltestperf19:51:052
3594790992919,8cyclictest421-21dbus-daemon19:46:212
359479099291,21cyclictest3714581-21ssh23:26:182
3594790992910,4cyclictest171rcu_preempt22:41:252
3594789992921,5cyclictest421-21dbus-daemon20:10:011
3594789992920,6cyclictest0-21swapper/123:21:051
3594789992918,3cyclictest1896942-21cups-browsed20:19:411
3594789992918,3cyclictest171rcu_preempt19:46:111
359479099285,6cyclictest62750lldpd22:00:562
359479099285,6cyclictest62750lldpd22:00:562
3594790992820,5cyclictest3654040-21unixbench_multi21:46:262
3594790992812,11cyclictest351ktimers/220:51:142
3594789992821,4cyclictest3609748-21cron20:00:011
3594789992816,3cyclictest171rcu_preempt20:46:251
3594789992815,3cyclictest171rcu_preempt19:11:081
3594788992819,6cyclictest3594800-21munin-run19:10:010
3594790992716,8cyclictest1001-21mta-sts-daemon21:05:022
3594789992716,3cyclictest171rcu_preempt21:01:021
3594789992716,3cyclictest171rcu_preempt20:11:151
3594789992716,2cyclictest171rcu_preempt20:21:141
3594790992621,3cyclictest0-21swapper/220:10:012
3594790992619,4cyclictest0-21swapper/200:31:282
3594789992616,3cyclictest171rcu_preempt19:55:011
3594790992518,4cyclictest421-21dbus-daemon20:32:112
3594790992516,5cyclictest3600770-21cron19:30:002
3594790992516,5cyclictest3600770-21cron19:30:002
359478999258,3cyclictest171rcu_preempt19:10:011
3594790992416,5cyclictest3610052-21tail20:01:032
3594790992116,2cyclictest0-21swapper/220:21:072
3594790992016,2cyclictest0-21swapper/219:10:012
359479199193,8cyclictest0-21swapper/319:16:143
359479199192,9cyclictest0-21swapper/321:41:143
3594790991913,3cyclictest0-21swapper/220:46:052
359479199181,9cyclictest0-21swapper/323:21:173
359479199181,11cyclictest0-21swapper/322:16:213
359479199181,10cyclictest0-21swapper/323:36:183
359479199176,4cyclictest0-21swapper/323:56:133
359479199172,9cyclictest0-21swapper/322:01:133
359479199172,9cyclictest0-21swapper/322:01:133
359479199172,10cyclictest0-21swapper/300:36:153
359479199172,10cyclictest0-21swapper/300:36:153
359479199171,9cyclictest0-21swapper/322:36:183
359479199171,9cyclictest0-21swapper/322:21:113
359479199171,8cyclictest0-21swapper/322:51:023
359479199171,8cyclictest0-21swapper/322:51:023
359479199171,8cyclictest0-21swapper/322:11:133
359479199171,7cyclictest0-21swapper/320:31:143
359479199171,10cyclictest0-21swapper/322:56:193
359479199171,10cyclictest0-21swapper/321:56:133
359479199162,9cyclictest0-21swapper/320:56:143
359479199162,8cyclictest0-21swapper/323:41:343
359479199161,9cyclictest0-21swapper/323:16:113
359479199161,8cyclictest0-21swapper/321:26:103
359479199161,8cyclictest0-21swapper/300:16:143
359479199161,7cyclictest0-21swapper/319:51:133
359479199161,10cyclictest0-21swapper/322:06:183
359479199161,10cyclictest0-21swapper/322:06:183
359479199152,8cyclictest0-21swapper/321:16:173
359479199152,7cyclictest0-21swapper/319:30:013
359479199152,7cyclictest0-21swapper/319:30:013
359479199152,7cyclictest0-21swapper/300:26:083
359479199152,6cyclictest0-21swapper/320:40:023
359479199152,6cyclictest0-21swapper/319:46:153
359479199151,9cyclictest0-21swapper/323:46:143
359479199151,9cyclictest0-21swapper/323:06:203
359479199151,9cyclictest0-21swapper/321:31:183
359479199151,9cyclictest0-21swapper/300:01:133
359479199151,8cyclictest0-21swapper/300:15:023
359479199151,7cyclictest0-21swapper/321:51:123
359479199151,7cyclictest0-21swapper/320:41:143
359479199151,6cyclictest0-21swapper/320:51:113
359479199151,6cyclictest0-21swapper/319:56:123
359479199151,6cyclictest0-21swapper/319:31:163
359479199151,6cyclictest0-21swapper/319:11:143
359479199146,5cyclictest0-21swapper/322:31:333
359479199146,5cyclictest0-21swapper/300:06:253
359479199146,4cyclictest0-21swapper/300:21:093
359479199142,8cyclictest0-21swapper/321:11:153
359479199142,7cyclictest0-21swapper/321:36:113
359479199141,8cyclictest0-21swapper/323:02:333
359479199141,8cyclictest0-21swapper/322:26:133
359479199141,7cyclictest0-21swapper/323:51:123
359479199141,7cyclictest0-21swapper/321:46:133
359479199141,5cyclictest0-21swapper/320:16:113
359479199140,8cyclictest0-21swapper/322:46:113
359479199140,8cyclictest0-21swapper/322:46:113
359479199138,3cyclictest0-21swapper/323:31:043
359479199137,4cyclictest0-21swapper/323:26:083
359479199136,5cyclictest0-21swapper/323:10:143
359479199136,5cyclictest0-21swapper/320:06:033
359479199136,4cyclictest0-21swapper/300:31:203
359479199135,4cyclictest0-21swapper/321:21:033
359479199131,7cyclictest0-21swapper/320:21:163
359479199131,7cyclictest0-21swapper/319:41:113
359479199131,6cyclictest0-21swapper/320:46:143
359479199131,6cyclictest0-21swapper/320:11:173
359479199131,6cyclictest0-21swapper/320:01:133
359479199131,6cyclictest0-21swapper/319:36:163
359479199131,6cyclictest0-21swapper/319:21:133
359479199126,4cyclictest0-21swapper/320:26:113
359479199125,5cyclictest0-21swapper/322:40:123
359479199121,6cyclictest0-21swapper/321:06:133
359479199116,3cyclictest0-21swapper/321:01:113
359479199104,3cyclictest0-21swapper/319:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional