You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-13 - 13:21

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Sat Jun 13, 2026 00:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
4045991997059,7cyclictest4135727-21grep22:35:011
4045990995447,5cyclictest4192467-21cron00:05:010
4045991995038,5cyclictest0-21swapper/122:20:241
4045992994925,17cyclictest750-21ntpd20:20:212
4045991994941,5cyclictest4164071-21munin-plugin-st23:20:011
4045992994837,5cyclictest0-21swapper/223:35:242
4045992994837,5cyclictest0-21swapper/223:35:242
4045991994841,4cyclictest1000-21mta-sts-daemon23:40:251
4045991994841,4cyclictest1000-21mta-sts-daemon23:40:251
4045990994838,5cyclictest1002-21mta-sts-daemon00:15:280
4045992994733,8cyclictest2299980-21cups-browsed21:50:022
4045992994710,7cyclictest171rcu_preempt21:40:292
4045990994737,7cyclictest4160900-21latency_hist23:15:030
404599099470,4cyclictest171rcu_preempt20:10:280
4045991994638,5cyclictest4081979-21munin-run21:10:011
4045991994637,6cyclictest4085504-21sh21:15:161
4045991994634,8cyclictest0-21swapper/123:05:021
404599199461,3cyclictest171rcu_preempt23:45:251
404599199461,3cyclictest171rcu_preempt23:45:251
4045990994631,12cyclictest4116787-21ssh22:05:010
4045992994536,6cyclictest0-21swapper/222:15:012
404599299453,33cyclictest9077-21ssh00:20:282
404599199452,37cyclictest5938-21ssh00:15:301
404599099453,35cyclictest2299980-21cups-browsed22:10:240
4045992994332,7cyclictest2299980-21cups-browsed23:16:182
4045992994328,12cyclictest6315-21timerwakeupswit00:15:362
4045991994335,5cyclictest4142034-21cron22:45:011
4045990994335,6cyclictest4143184-21ssh22:45:310
4045990994335,5cyclictest420-21avahi-daemon21:35:320
4045992994233,5cyclictest421-21dbus-daemon19:20:022
404599299422,30cyclictest12175-21ssh00:25:262
4045991994228,11cyclictest4146549-21switchtime22:50:401
4045990994235,5cyclictest4154625-21munin-plugin-st23:05:010
4045990994229,11cyclictest14-21ksoftirqd/022:25:240
404599099422,7cyclictest171rcu_preempt22:50:280
404599099421,38cyclictest665-21snmpd19:20:270
404599299414,28cyclictest4184066-21ssh23:50:272
404599299414,28cyclictest4184066-21ssh23:50:272
4045992994133,5cyclictest4170454-21latency_hist23:30:012
4045992994133,5cyclictest4059494-21systemd-run19:55:012
404599099411,37cyclictest0-21swapper/023:25:260
4045992994032,3cyclictest0-21swapper/223:00:362
4045992994031,6cyclictest4101011-21awk21:40:012
4045991994032,5cyclictest11152-21systemd-run00:25:011
404599099404,27cyclictest4139873-21ssh22:40:230
4045990994031,7cyclictest151ktimers/000:35:260
4045990994030,7cyclictest151ktimers/023:45:250
4045990994030,7cyclictest151ktimers/023:45:250
4045990994026,11cyclictest12650irq/32-eth023:30:270
4045990994021,3cyclictest171rcu_preempt21:00:280
404599099401,28cyclictest4162032-21ssh23:15:290
4045992993930,5cyclictest4094690-21systemd-run21:30:012
404599199395,30cyclictest4114694-21ssh22:00:261
4045991993931,5cyclictest4081977-21diskmemload23:15:021
4045991993931,5cyclictest4054981-21awk19:40:011
4045990993933,4cyclictest4189280-21systemctl00:00:000
4045990993932,5cyclictest4083516-21grep21:10:410
4045990993930,7cyclictest4110482-21sh21:55:010
404599299386,7cyclictest4081977-21diskmemload21:55:322
4045992993831,4cyclictest4182992-21systemd-run23:50:012
4045992993831,4cyclictest4182992-21systemd-run23:50:012
4045992993829,5cyclictest0-21swapper/222:25:272
404599199385,28cyclictest4089361-21ssh21:20:251
404599199384,26cyclictest4189302-21awk00:00:011
404599199383,30cyclictest4083156-21ssh21:10:361
4045991993828,7cyclictest20570-21sendmail00:40:011
4045991993823,8cyclictest0-21swapper/122:05:271
404599299372,25cyclictest4139918-21ssh22:40:252
404599199377,6cyclictest28-21ksoftirqd/123:30:241
404599199373,26cyclictest4143091-21ssh22:45:261
4045991993730,4cyclictest487-21rs:main1
4045991993725,9cyclictest4168362-21ssh23:25:301
4045991993725,9cyclictest4061882-21ntp_kernel_pll_20:00:251
4045991993725,9cyclictest4061882-21ntp_kernel_pll_20:00:251
4045990993729,5cyclictest1-21systemd23:25:010
4045990993727,4cyclictest0-21swapper/021:40:290
404599299368,18cyclictest2778-21ssh00:10:302
404599299365,23cyclictest4095671-21ssh21:30:242
4045992993630,3cyclictest0-21swapper/219:40:012
4045992993628,5cyclictest4156186-21grep23:05:372
4045992993628,5cyclictest4060956-21cron20:00:002
4045992993628,5cyclictest4060956-21cron20:00:002
4045992993625,8cyclictest4110667-21latency_hist21:55:022
4045992993619,4cyclictest171rcu_preempt20:10:282
4045991993627,5cyclictest17395-21cron00:35:011
4045991993621,9cyclictest567-21gdbus21:55:341
404599199362,11cyclictest28-21ksoftirqd/120:45:281
4045991993618,14cyclictest271ktimers/120:30:241
404599099364,6cyclictest171rcu_preempt20:00:270
404599099364,6cyclictest171rcu_preempt20:00:270
404599299358,15cyclictest4193508-21ssh00:05:292
4045992993529,3cyclictest0-21swapper/222:15:282
4045992993524,7cyclictest4187790-21sh23:55:402
404599199354,27cyclictest0-21swapper/120:50:281
404599199354,17cyclictest4120956-21ssh22:10:241
4045991993528,4cyclictest4105103-21awk21:45:211
4045991993526,5cyclictest4176043-21kworker/u8:4-rpciod23:39:491
4045991993526,5cyclictest4176043-21kworker/u8:4-rpciod23:39:491
4045991993522,3cyclictest171rcu_preempt23:05:341
404599099358,7cyclictest171rcu_preempt19:15:220
404599099355,26cyclictest4184020-21ssh23:50:260
404599099355,26cyclictest4184020-21ssh23:50:260
4045990993531,2cyclictest0-21swapper/020:25:220
4045990993528,4cyclictest4138882-21grep22:40:010
404599099352,27cyclictest12118-21ssh00:25:230
4045990993515,12cyclictest4086213-21sh21:15:300
4045992993428,3cyclictest0-21swapper/200:00:172
4045992993426,5cyclictest4179862-21latency_hist23:45:012
4045992993426,5cyclictest4179862-21latency_hist23:45:012
4045992993426,5cyclictest4071396-21unixbench_singl20:30:362
4045991993427,3cyclictest0-21swapper/121:25:281
4045991993425,6cyclictest4078958-21idleruntime-cro21:00:011
404599199342,22cyclictest4149368-21ssh22:55:271
404599199341,26cyclictest4102069-21ssh21:40:271
404599099346,20cyclictest15284-21ssh00:30:250
4045990993428,3cyclictest0-21swapper/000:05:190
4045990993427,4cyclictest4151466-21ls23:00:010
4045990993426,6cyclictest4093045-21unixbench_multi21:25:340
4045990993426,5cyclictest12750irq/33-eth022:30:250
4045990993425,7cyclictest4066279-21needreboot20:15:210
4045990993414,13cyclictest4127240-21ssh22:20:230
404599099340,3cyclictest171rcu_preempt23:05:270
404599299339,18cyclictest351ktimers/220:15:282
4045992993326,4cyclictest4148329-21idleruntime-cro22:55:012
4045992993325,4cyclictest0-21swapper/220:55:012
4045992993321,6cyclictest0-21swapper/220:05:272
4045991993327,3cyclictest0-21swapper/119:25:011
4045991993325,5cyclictest4167219-21cron23:25:001
4045991993325,5cyclictest4063007-21fschecks_count20:05:181
4045991993325,5cyclictest3180-21unixbench-2d00:10:361
4045991993321,8cyclictest0-21swapper/100:00:321
4045991993320,11cyclictest271ktimers/119:10:271
4045990993310,18cyclictest2299980-21cups-browsed19:35:280
4045992993224,5cyclictest420-21avahi-daemon23:10:262
4045992993224,5cyclictest4170980-21ssh23:30:172
4045992993224,5cyclictest4149507-21proc_pri22:55:332
4045992993224,5cyclictest4136323-21sort22:35:162
404599199329,9cyclictest28-21ksoftirqd/120:20:241
404599199326,9cyclictest28-21ksoftirqd/121:00:291
404599199324,20cyclictest4136696-21ntpq22:35:211
404599199321,8cyclictest261rcuc/100:25:301
4045991993216,8cyclictest4081977-21diskmemload21:35:251
4045991993215,3cyclictest171rcu_preempt19:25:341
4045991993215,12cyclictest627-21lldpd00:09:391
4045990993226,4cyclictest4119973-21systemd-run22:10:010
4045990993223,5cyclictest0-21swapper/021:05:260
4045990993220,7cyclictest0-21swapper/023:40:000
4045990993220,7cyclictest0-21swapper/023:40:000
404599299316,15cyclictest4143114-21ssh22:45:272
4045992993122,5cyclictest4073020-21latency_hist20:40:022
4045992993121,7cyclictest4133450-21missed_timers22:30:182
404599199318,5cyclictest28-21ksoftirqd/120:00:011
404599199318,5cyclictest28-21ksoftirqd/120:00:011
4045991993125,3cyclictest0-21swapper/123:55:011
4045991993125,3cyclictest0-21swapper/123:55:011
4045991993123,5cyclictest4071699-21sshd20:35:131
4045991993123,5cyclictest2299980-21cups-browsed21:50:241
404599199312,24cyclictest0-21swapper/120:25:201
4045991993118,3cyclictest171rcu_preempt19:50:281
404599199311,7cyclictest28-21ksoftirqd/120:15:231
404599099313,5cyclictest171rcu_preempt20:40:300
4045990993124,3cyclictest0-21swapper/021:45:260
4045990993122,7cyclictest2299980-21cups-browsed19:25:340
404599299304,17cyclictest1003-21mta-sts-daemon21:20:222
4045992993023,3cyclictest0-21swapper/221:05:012
4045992993022,5cyclictest4117541-21kernelversion22:05:192
404599299301,24cyclictest4086095-21ssh21:15:242
404599299301,21cyclictest18532-21ssh00:35:292
404599199303,19cyclictest4124245-21ssh22:15:311
4045991993024,3cyclictest0-21swapper/121:30:181
4045991993014,3cyclictest171rcu_preempt19:45:021
404599099305,6cyclictest171rcu_preempt20:55:300
4045990993025,3cyclictest4067220-21kworker/u8:10-xprtiod23:45:010
4045990993025,3cyclictest4067220-21kworker/u8:10-xprtiod23:45:010
4045990993022,4cyclictest0-21swapper/021:30:160
4045990993022,4cyclictest0-21swapper/000:10:290
4045990993012,13cyclictest421-21dbus-daemon19:40:250
404599299298,5cyclictest171rcu_preempt19:10:272
4045992992923,3cyclictest0-21swapper/221:15:012
404599299292,23cyclictest4165099-21ssh23:20:242
4045991992919,6cyclictest4048040-21/usr/sbin/munin19:15:181
4045991992914,3cyclictest171rcu_preempt20:10:171
4045992992822,3cyclictest0-21swapper/220:05:002
4045992992822,3cyclictest0-21swapper/220:05:002
4045992992820,5cyclictest15390-21ntpq00:30:302
4045991992812,3cyclictest171rcu_preempt19:30:261
4045991992810,3cyclictest171rcu_preempt20:40:231
4045990992822,3cyclictest0-21swapper/021:55:260
4045990992822,3cyclictest0-21swapper/020:30:270
4045990992821,4cyclictest12650irq/32-eth022:15:310
4045990992820,5cyclictest4089489-21proc_pri21:20:300
4045990992819,7cyclictest4072907-21unixbench_singl20:35:410
404599299272,20cyclictest4114604-21ssh22:00:222
4045992992720,4cyclictest4127660-21threads22:20:362
4045992992720,4cyclictest4051123-21kernelversion19:25:222
404599199273,6cyclictest28-21ksoftirqd/119:45:181
404599099272,19cyclictest9117-21sh00:20:300
4045990992719,6cyclictest4075653-21switchtime20:45:360
4045992992619,4cyclictest4074382-21unixbench_multi20:40:362
4045992992618,6cyclictest0-21swapper/221:00:012
4045992992617,6cyclictest4056871-21basename19:45:152
4045990992621,3cyclictest1001-21mta-sts-daemon19:30:370
404599099260,3cyclictest171rcu_preempt19:45:220
4045992992517,5cyclictest439-21systemd-logind20:30:012
4045990992514,5cyclictest151ktimers/019:50:240
404599099250,15cyclictest0-21swapper/019:10:290
4045992992417,4cyclictest1001-21mta-sts-daemon19:20:322
4045992992416,5cyclictest4053415-21unixbench_singl19:30:362
4045991992411,2cyclictest171rcu_preempt19:10:011
404599099242,4cyclictest171rcu_preempt20:50:250
404599099242,3cyclictest567-21gdbus19:59:520
404599099242,3cyclictest567-21gdbus19:59:520
4045990992415,8cyclictest12650irq/32-eth020:20:200
4045992992317,3cyclictest0-21swapper/219:45:012
4045992992314,6cyclictest4081855-21grep21:05:362
404599099233,5cyclictest171rcu_preempt20:05:280
404599299222,16cyclictest2299980-21cups-browsed20:45:252
404599099212,13cyclictest4045947-21munin-run19:10:010
404599399193,7cyclictest0-21swapper/322:10:243
4045992991914,2cyclictest0-21swapper/219:10:032
404599399182,6cyclictest0-21swapper/321:05:273
404599399173,5cyclictest0-21swapper/320:10:303
404599399172,8cyclictest0-21swapper/323:05:273
404599399172,8cyclictest0-21swapper/323:00:233
404599399172,8cyclictest0-21swapper/322:35:253
404599399171,4cyclictest0-21swapper/300:15:253
404599399164,8cyclictest0-21swapper/300:25:223
404599399163,7cyclictest0-21swapper/322:30:283
404599399163,7cyclictest0-21swapper/322:20:233
404599399162,8cyclictest0-21swapper/321:45:273
404599399162,8cyclictest0-21swapper/321:35:253
404599399162,8cyclictest0-21swapper/321:20:283
404599399162,6cyclictest0-21swapper/319:55:253
404599399162,6cyclictest0-21swapper/319:55:253
404599399162,6cyclictest0-21swapper/319:30:243
404599399162,5cyclictest0-21swapper/321:00:233
404599399161,6cyclictest0-21swapper/319:45:253
404599399153,7cyclictest0-21swapper/323:50:243
404599399153,7cyclictest0-21swapper/323:50:243
404599399153,5cyclictest0-21swapper/320:50:243
404599399152,8cyclictest0-21swapper/321:30:163
404599399152,7cyclictest0-21swapper/321:40:303
404599399152,6cyclictest0-21swapper/320:05:273
404599399151,8cyclictest0-21swapper/323:40:363
404599399151,8cyclictest0-21swapper/323:40:363
404599399151,8cyclictest0-21swapper/321:25:253
404599399151,8cyclictest0-21swapper/300:00:353
404599399151,7cyclictest0-21swapper/319:15:303
404599399151,6cyclictest0-21swapper/320:40:263
404599399151,6cyclictest0-21swapper/319:10:223
404599399144,7cyclictest0-21swapper/323:55:253
404599399144,6cyclictest0-21swapper/323:15:223
404599399144,5cyclictest0-21swapper/319:25:303
404599399144,4cyclictest0-21swapper/322:45:233
404599399143,5cyclictest0-21swapper/322:00:223
404599399142,8cyclictest0-21swapper/320:30:283
404599399142,8cyclictest0-21swapper/300:30:223
404599399142,7cyclictest0-21swapper/323:25:273
404599399142,7cyclictest0-21swapper/321:50:273
404599399142,6cyclictest0-21swapper/300:35:233
404599399142,5cyclictest0-21swapper/320:55:263
404599399141,8cyclictest0-21swapper/321:15:303
404599399141,7cyclictest0-21swapper/320:35:283
404599399141,6cyclictest0-21swapper/320:20:243
404599399141,6cyclictest0-21swapper/300:05:333
404599399141,3cyclictest0-21swapper/320:25:203
404599399134,5cyclictest0-21swapper/323:20:243
404599399133,6cyclictest0-21swapper/321:10:283
404599399132,8cyclictest0-21swapper/300:20:253
404599399132,7cyclictest0-21swapper/322:05:273
404599399132,6cyclictest0-21swapper/323:45:223
404599399132,6cyclictest0-21swapper/323:45:223
404599399132,6cyclictest0-21swapper/322:55:243
404599399132,6cyclictest0-21swapper/319:20:253
404599399131,8cyclictest0-21swapper/323:10:273
404599399131,7cyclictest0-21swapper/323:35:273
404599399131,7cyclictest0-21swapper/323:35:273
404599399131,7cyclictest0-21swapper/322:50:293
404599399131,7cyclictest0-21swapper/322:15:223
404599399131,6cyclictest0-21swapper/320:45:253
404599399131,5cyclictest0-21swapper/319:35:223
404599399130,6cyclictest0-21swapper/320:15:233
404599399121,7cyclictest0-21swapper/322:25:203
404599399121,7cyclictest0-21swapper/321:55:273
404599399121,6cyclictest0-21swapper/322:40:213
404599399121,6cyclictest0-21swapper/320:00:253
404599399121,6cyclictest0-21swapper/320:00:253
404599399121,5cyclictest0-21swapper/300:10:093
404599399120,7cyclictest0-21swapper/323:30:243
404599399111,6cyclictest0-21swapper/319:50:233
404599399111,5cyclictest0-21swapper/319:40:173
40459939981,4cyclictest0-21swapper/319:10:033
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional