You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-17 - 07:24

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #4, slot #1

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1s.osadl.org (updated Sun May 17, 2026 00:44:17)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1153221300,3sleep30-21swapper/320:02:303
1153221300,3sleep30-21swapper/320:02:303
2499621010,3sleep00-21swapper/022:02:470
2499621010,3sleep00-21swapper/022:02:470
2499621010,3sleep00-21swapper/022:02:470
281562930,1chrt28153-21open_files23:37:471
281562930,1chrt28153-21open_files23:37:471
11132890,1sleep01114-21/usr/sbin/munin19:32:520
11132890,1sleep01114-21/usr/sbin/munin19:32:520
125932740,2chrt12590-21ntpq00:22:461
125932740,2chrt12590-21ntpq00:22:461
125932740,2chrt12590-21ntpq00:22:461
114212740,2chrt11424-21mailstats22:52:491
114212740,2chrt11424-21mailstats22:52:491
295552610,3sleep00-21swapper/020:47:500
295552610,3sleep00-21swapper/020:47:500
295552610,3sleep00-21swapper/020:47:500
288482580,14sleep0201rcuc/019:22:270
295312560,3chrt29533-21sort23:42:410
295312560,3chrt29533-21sort23:42:410
25548995244,4cyclictest4680-21head00:02:390
25548995244,4cyclictest4680-21head00:02:390
25548995244,4cyclictest4680-21head00:02:390
132152520,2sleep013216-21fschecks_time21:32:400
25548994820,24cyclictest30103-21kworker/u9:4+rpciod00:02:320
25548994820,24cyclictest30103-21kworker/u9:4+rpciod00:02:320
72062460,3sleep20-21swapper/219:47:502
72062460,3sleep20-21swapper/219:47:502
25548994337,4cyclictest19136-21cat00:42:270
25548994337,4cyclictest19136-21cat00:42:270
177202410,3chrt0-21swapper/220:17:412
25556994021,4cyclictest11851-21kworker/u9:1-xprtiod00:03:191
25556994021,4cyclictest11851-21kworker/u9:1-xprtiod00:03:191
25556994021,4cyclictest11851-21kworker/u9:1-xprtiod00:03:191
129552400,3sleep20-21swapper/221:32:382
129552400,3sleep20-21swapper/221:32:382
303032390,3sleep30-21swapper/319:22:503
303032390,3sleep30-21swapper/319:22:503
218532390,2sleep121842-21fschecks_count23:22:401
218532390,2sleep121842-21fschecks_count23:22:401
218532390,2sleep121842-21fschecks_count23:22:401
103592390,3sleep20-21swapper/219:57:432
103592390,3sleep20-21swapper/219:57:432
25556993828,7cyclictest31390-21timerandwakeup20:52:481
25556993828,7cyclictest31390-21timerandwakeup20:52:481
25548993829,7cyclictest16842-21netstat23:07:440
25548993829,7cyclictest16842-21netstat23:07:440
141442370,2sleep00-21swapper/000:27:430
141442370,2sleep00-21swapper/000:27:430
25556993527,6cyclictest4844-21netstat19:42:451
25556993527,6cyclictest4844-21netstat19:42:451
25556993526,7cyclictest26429-21sendmail20:42:271
25556993526,7cyclictest26429-21sendmail20:42:271
25548993528,5cyclictest23848-21fschecks_time23:27:400
25548993528,5cyclictest23848-21fschecks_time23:27:400
25556993424,7cyclictest5320-21latency_hist21:12:301
25556993424,7cyclictest5320-21latency_hist21:12:301
25548993423,9cyclictest12677-21processes00:22:480
25548993423,9cyclictest12677-21processes00:22:480
25548993423,9cyclictest12677-21processes00:22:480
2556399331,22cyclictest4233-21taskset00:02:332
2556399331,22cyclictest4233-21taskset00:02:332
25556993325,6cyclictest547-21rs:main1
25556993325,6cyclictest547-21rs:main1
243842330,3chrt0-21swapper/223:27:472
243842330,3chrt0-21swapper/223:27:472
227922330,2sleep20-21swapper/221:57:422
227922330,2sleep20-21swapper/221:57:422
25556993225,5cyclictest15784-21cut00:32:411
25556993225,5cyclictest15784-21cut00:32:411
25556993225,5cyclictest15784-21cut00:32:411
25556993224,6cyclictest18493-21threads00:37:501
25556993224,6cyclictest18493-21threads00:37:501
25556993224,6cyclictest12574-21fschecks_time22:57:411
25556993224,6cyclictest12574-21fschecks_time22:57:411
25556993223,7cyclictest4620-21cron22:37:271
25556993223,7cyclictest4620-21cron22:37:271
145592320,3sleep014741-21unixbench_multi20:07:510
25556993024,4cyclictest29892-21missed_timers23:42:441
25556993024,4cyclictest29892-21missed_timers23:42:441
25548993025,3cyclictest0-21swapper/022:22:270
25548993023,5cyclictest26959-21sshd23:35:240
25548993023,5cyclictest26959-21sshd23:35:240
25556992925,3cyclictest27383-21ntp_offset20:42:461
25556992925,3cyclictest27383-21ntp_offset20:42:461
25556992922,5cyclictest550-21cron19:57:271
25556992922,5cyclictest550-21cron19:57:271
25548992921,6cyclictest4692-21latency_hist22:37:300
25548992921,6cyclictest4692-21latency_hist22:37:300
25556992823,4cyclictest25535-21cyclictest21:57:421
25556992823,4cyclictest25535-21cyclictest21:57:421
25556992823,2cyclictest0-21swapper/120:52:271
25556992823,2cyclictest0-21swapper/120:52:271
25556992823,2cyclictest0-21swapper/120:52:271
25556992822,5cyclictest13003-21cpuspeed21:32:381
25556992822,5cyclictest13003-21cpuspeed21:32:381
25556992821,5cyclictest14678-21systemd-journal22:52:271
25556992821,5cyclictest14678-21systemd-journal22:52:271
25556992821,5cyclictest14678-21systemd-journal22:52:271
25548992823,4cyclictest545-21in:imuxsock20:02:270
25548992823,4cyclictest545-21in:imuxsock20:02:270
25548992823,4cyclictest32180-21kernelversion22:22:420
25548992823,4cyclictest32180-21kernelversion22:22:420
25548992823,4cyclictest32180-21kernelversion22:22:420
25548992822,4cyclictest7874-21grep00:12:270
25548992822,4cyclictest7874-21grep00:12:270
25548992822,4cyclictest7874-21grep00:12:270
25548992821,5cyclictest5798-21sshd19:47:250
25548992821,5cyclictest5798-21sshd19:47:250
25548992821,5cyclictest21343-21sshd23:20:240
25548992821,5cyclictest21343-21sshd23:20:240
25548992821,5cyclictest21343-21sshd23:20:240
2556399276,17cyclictest326102chrt23:50:192
2556399276,17cyclictest326102chrt23:50:192
2556399276,17cyclictest326102chrt23:50:192
2556399271,3cyclictest324122sleep219:30:092
2556399271,3cyclictest324122sleep219:30:092
25556992722,3cyclictest0-21swapper/100:02:301
25556992722,3cyclictest0-21swapper/100:02:301
25556992721,5cyclictest5597-21fschecks_count21:12:401
25556992720,5cyclictest1648-21ntpd22:12:461
25556992720,5cyclictest1648-21ntpd22:12:461
25556992720,5cyclictest1648-21ntpd22:12:461
25548992723,2cyclictest0-21swapper/021:02:400
25548992723,2cyclictest0-21swapper/021:02:400
25548992723,2cyclictest0-21swapper/021:02:400
25548992722,3cyclictest0-21swapper/021:57:410
25548992722,3cyclictest0-21swapper/021:57:410
25548992721,5cyclictest6288-21fschecks_time19:47:400
25548992721,5cyclictest6288-21fschecks_time19:47:400
25548992721,3cyclictest0-21swapper/020:07:270
25548992721,3cyclictest0-21swapper/020:07:270
25548992720,4cyclictest0-21swapper/022:52:390
25548992720,4cyclictest0-21swapper/022:52:390
25548992719,6cyclictest16825-21unixbench_multi00:32:510
25548992719,6cyclictest16825-21unixbench_multi00:32:510
25548992719,6cyclictest16825-21unixbench_multi00:32:510
25548992719,5cyclictest4265-21ntp_kernel_err21:07:470
25548992719,5cyclictest4265-21ntp_kernel_err21:07:470
263262260,2sleep30-21swapper/322:07:403
2556399267,15cyclictest298112chrt20:48:162
2556399267,15cyclictest298112chrt20:48:162
2556399267,15cyclictest298112chrt20:48:162
25556992622,3cyclictest25535-21cyclictest22:11:251
25556992622,3cyclictest25535-21cyclictest22:11:251
25556992622,3cyclictest25535-21cyclictest22:11:251
25556992622,2cyclictest0-21swapper/122:47:271
25556992622,2cyclictest0-21swapper/122:47:271
25556992622,2cyclictest0-21swapper/121:42:271
25556992622,2cyclictest0-21swapper/121:42:271
25556992621,3cyclictest0-21swapper/119:57:401
25556992621,3cyclictest0-21swapper/119:57:401
25556992619,5cyclictest19663-21kernelversion20:22:421
25556992619,5cyclictest19663-21kernelversion20:22:421
25556992615,9cyclictest14678-21systemd-journal19:22:271
25548992622,3cyclictest29870-21cpuspeed22:17:380
25548992622,3cyclictest29870-21cpuspeed22:17:380
25548992621,3cyclictest10209-21cut00:17:410
25548992621,3cyclictest10209-21cut00:17:410
25548992621,3cyclictest10209-21cut00:17:410
25548992619,5cyclictest19338-21fschecks_count20:22:400
25548992619,5cyclictest19338-21fschecks_count20:22:400
25548992619,5cyclictest11526-21timerandwakeup22:52:500
25548992619,5cyclictest11526-21timerandwakeup22:52:500
25548992618,6cyclictest32045-21ls19:27:510
25548992618,6cyclictest32045-21ls19:27:510
25548992618,6cyclictest17666-21cut21:42:490
25548992618,6cyclictest17666-21cut21:42:490
25548992613,11cyclictest9552-21cron19:57:270
25548992613,11cyclictest9552-21cron19:57:270
2556399251,7cyclictest16368-21chrt21:40:582
2556399251,7cyclictest16368-21chrt21:40:582
2556399251,7cyclictest10752-21chrt21:25:552
2556399251,7cyclictest10752-21chrt21:25:552
2556399251,4cyclictest0-21swapper/220:22:512
2556399251,4cyclictest0-21swapper/220:22:512
2556399251,3cyclictest0-21swapper/223:18:562
2556399251,3cyclictest0-21swapper/223:18:562
2556399251,3cyclictest0-21swapper/223:18:562
2556399251,3cyclictest0-21swapper/220:44:002
2556399251,3cyclictest0-21swapper/220:44:002
25556992521,2cyclictest0-21swapper/120:17:391
25556992521,2cyclictest0-21swapper/119:42:271
25556992521,2cyclictest0-21swapper/119:42:271
25556992520,3cyclictest0-21swapper/121:42:411
25556992520,3cyclictest0-21swapper/121:42:411
25556992520,3cyclictest0-21swapper/121:32:501
25556992519,4cyclictest0-21swapper/122:07:411
25556992518,4cyclictest0-21swapper/121:52:271
25556992518,4cyclictest0-21swapper/121:52:271
25556992517,5cyclictest1447-21sendmail_mailqu23:52:481
25556992517,5cyclictest1447-21sendmail_mailqu23:52:481
25548992521,2cyclictest0-21swapper/022:42:430
25548992521,2cyclictest0-21swapper/022:42:430
25548992520,4cyclictest14678-21systemd-journal00:17:240
25548992520,4cyclictest14678-21systemd-journal00:17:240
25548992520,3cyclictest0-21swapper/023:12:430
25548992520,3cyclictest0-21swapper/023:12:430
25548992519,5cyclictest20949-21kernelversion21:52:410
25548992519,5cyclictest20949-21kernelversion21:52:410
25548992519,5cyclictest1454-21latency_hist21:02:270
25548992519,5cyclictest1454-21latency_hist21:02:270
25548992519,4cyclictest90-1pr/ttyS023:52:320
25548992519,4cyclictest90-1pr/ttyS023:52:320
25548992519,4cyclictest90-1pr/ttyS023:52:320
25548992518,5cyclictest26423-21latency_hist20:42:270
25548992518,5cyclictest26423-21latency_hist20:42:270
25548992518,5cyclictest25535-21cyclictest20:27:430
25548992518,5cyclictest25535-21cyclictest20:27:430
25548992517,6cyclictest8031-21ntp_kernel_err21:17:470
25548992517,6cyclictest8031-21ntp_kernel_err21:17:470
2556399248,13cyclictest30062chrt21:03:192
2556399248,13cyclictest30062chrt21:03:192
2556399248,13cyclictest30062chrt21:03:192
2556399247,13cyclictest119722chrt22:55:192
2556399247,13cyclictest119722chrt22:55:192
2556399241,8cyclictest6947-21chrt21:15:332
2556399241,7cyclictest4935-21chrt22:37:382
2556399241,7cyclictest4935-21chrt22:37:382
2556399241,7cyclictest15567-21chrt23:02:592
2556399241,7cyclictest15567-21chrt23:02:592
2556399241,4cyclictest36522chrt19:39:212
2556399241,4cyclictest36522chrt19:39:212
2556399241,4cyclictest0-21swapper/223:22:472
2556399241,4cyclictest0-21swapper/223:22:472
2556399241,4cyclictest0-21swapper/220:17:332
2556399241,4cyclictest0-21swapper/220:17:332
2556399241,3cyclictest219792sleep221:56:022
2556399241,3cyclictest219792sleep221:56:022
2556399241,3cyclictest0-21swapper/223:32:552
2556399241,3cyclictest0-21swapper/223:32:552
2556399241,3cyclictest0-21swapper/222:46:402
2556399241,3cyclictest0-21swapper/222:46:402
2556399241,3cyclictest0-21swapper/219:57:322
2556399241,3cyclictest0-21swapper/219:57:322
2556399241,3cyclictest0-21swapper/200:34:502
2556399241,3cyclictest0-21swapper/200:34:502
2556399241,3cyclictest0-21swapper/200:34:502
25556992419,3cyclictest0-21swapper/123:27:281
25556992419,3cyclictest0-21swapper/123:27:281
25556992418,4cyclictest3464-21latency_hist21:07:331
25556992418,4cyclictest3464-21latency_hist21:07:331
25556992418,4cyclictest3464-21latency_hist21:07:331
25556992418,4cyclictest25535-21cyclictest22:27:461
25556992418,4cyclictest25535-21cyclictest22:27:461
25556992417,5cyclictest550-21cron23:32:271
25556992417,5cyclictest550-21cron23:32:271
25556992417,5cyclictest30646-21sshd19:26:241
25556992417,5cyclictest30646-21sshd19:26:241
25556992416,6cyclictest26499-21threads23:32:501
25556992416,6cyclictest26499-21threads23:32:501
25548992420,2cyclictest0-21swapper/022:37:490
25548992420,2cyclictest0-21swapper/022:37:490
2554899241,9cyclictest2288-21grep22:27:520
2554899241,9cyclictest2288-21grep22:27:520
25548992419,3cyclictest30663-21tail20:52:400
25548992419,3cyclictest30663-21tail20:52:400
25548992419,3cyclictest0-21swapper/023:52:430
25548992419,3cyclictest0-21swapper/023:52:430
25548992419,3cyclictest0-21swapper/022:12:410
2556399233,15cyclictest0-21swapper/220:27:472
2556399233,15cyclictest0-21swapper/220:27:472
2556399232,3cyclictest0-21swapper/200:02:452
2556399232,3cyclictest0-21swapper/200:02:452
2556399232,3cyclictest0-21swapper/200:02:452
2556399231,9cyclictest28535-21chrt19:18:352
2556399231,8cyclictest8891-21chrt21:21:322
2556399231,8cyclictest8891-21chrt21:21:322
2556399231,8cyclictest11575-21chrt00:21:542
2556399231,8cyclictest11575-21chrt00:21:542
2556399231,8cyclictest11575-21chrt00:21:542
2556399231,7cyclictest5477-21chrt19:43:152
2556399231,7cyclictest5477-21chrt19:43:152
2556399231,7cyclictest31312-21chrt22:20:502
2556399231,6cyclictest9531-21chrt00:14:392
2556399231,6cyclictest9531-21chrt00:14:392
2556399231,6cyclictest6470-21chrt22:42:002
2556399231,6cyclictest6470-21chrt22:42:002
2556399231,6cyclictest14325-21chrt20:07:482
2556399231,4cyclictest7523-21chrt00:08:072
2556399231,4cyclictest7523-21chrt00:08:072
2556399231,4cyclictest7523-21chrt00:08:072
2556399231,4cyclictest4912-21chrt21:08:542
2556399231,4cyclictest4912-21chrt21:08:542
2556399231,4cyclictest31711-21chrt20:53:422
2556399231,4cyclictest31711-21chrt20:53:422
2556399231,4cyclictest27464-21chrt22:09:242
2556399231,4cyclictest27464-21chrt22:09:242
2556399231,4cyclictest27464-21chrt22:09:242
2556399231,4cyclictest263312sleep220:41:402
2556399231,4cyclictest263312sleep220:41:402
2556399231,4cyclictest20185-21chrt21:51:562
2556399231,4cyclictest20185-21chrt21:51:562
2556399231,4cyclictest18788-21chrt00:38:152
2556399231,4cyclictest18788-21chrt00:38:152
2556399231,4cyclictest13385-21chrt00:25:562
2556399231,4cyclictest13385-21chrt00:25:562
2556399231,4cyclictest13385-21chrt00:25:562
2556399231,4cyclictest130802chrt20:04:422
2556399231,4cyclictest130802chrt20:04:422
2556399231,4cyclictest10081-21chrt22:49:562
2556399231,4cyclictest10081-21chrt22:49:562
2556399231,4cyclictest10081-21chrt22:49:562
2556399231,3cyclictest243912sleep220:35:462
2556399231,3cyclictest243912sleep220:35:462
2556399231,3cyclictest195082sleep223:15:492
2556399231,3cyclictest195082sleep223:15:492
2556399231,3cyclictest174622sleep223:08:262
2556399231,3cyclictest174622sleep223:08:262
2556399231,3cyclictest142982sleep221:33:182
2556399231,3cyclictest0-21swapper/223:38:302
2556399231,3cyclictest0-21swapper/223:38:302
2556399231,3cyclictest0-21swapper/222:13:072
2556399231,3cyclictest0-21swapper/222:13:072
2556399231,3cyclictest0-21swapper/222:13:072
2556399231,3cyclictest0-21swapper/221:42:442
2556399231,3cyclictest0-21swapper/221:42:442
2556399231,3cyclictest0-21swapper/220:57:452
2556399231,3cyclictest0-21swapper/220:57:452
2556399231,3cyclictest0-21swapper/219:24:062
2556399231,3cyclictest0-21swapper/219:24:062
2556399231,3cyclictest0-21swapper/200:28:502
2556399231,3cyclictest0-21swapper/200:28:502
2556399231,2cyclictest0-21swapper/222:27:262
2556399231,2cyclictest0-21swapper/222:27:262
2556399231,2cyclictest0-21swapper/222:27:262
2556399231,2cyclictest0-21swapper/219:34:162
2556399231,2cyclictest0-21swapper/219:34:162
25556992318,4cyclictest749-21runrttasks19:27:521
25556992318,4cyclictest749-21runrttasks19:27:521
25556992318,4cyclictest6516-21idleruntime-cro22:42:281
25556992318,4cyclictest6516-21idleruntime-cro22:42:281
25556992318,3cyclictest0-21swapper/121:57:391
25556992318,3cyclictest0-21swapper/121:57:391
25556992317,4cyclictest1616-21sendmail-mta21:17:421
25556992317,4cyclictest1616-21sendmail-mta21:17:421
25556992317,4cyclictest0-21swapper/123:07:391
25556992317,4cyclictest0-21swapper/123:07:391
25556992316,5cyclictest32735-21timerandwakeup22:22:501
25556992316,5cyclictest32735-21timerandwakeup22:22:501
25556992316,5cyclictest32735-21timerandwakeup22:22:501
25556992314,7cyclictest10237-21fschecks_time00:17:411
25556992314,7cyclictest10237-21fschecks_time00:17:411
25556992314,7cyclictest10237-21fschecks_time00:17:411
25548992318,3cyclictest0-21swapper/023:02:430
25548992318,3cyclictest0-21swapper/023:02:430
25548992318,3cyclictest0-21swapper/022:07:510
25548992318,3cyclictest0-21swapper/022:07:510
25548992318,3cyclictest0-21swapper/022:07:510
25548992318,3cyclictest0-21swapper/020:17:460
25548992317,4cyclictest2589-21gltestperf19:37:400
25548992317,4cyclictest2589-21gltestperf19:37:400
25548992317,4cyclictest0-21swapper/022:57:510
25548992317,4cyclictest0-21swapper/022:57:510
25548992317,4cyclictest0-21swapper/021:47:410
25548992317,4cyclictest0-21swapper/021:47:410
25548992317,3cyclictest0-21swapper/021:37:400
25548992317,3cyclictest0-21swapper/021:37:400
25548992316,5cyclictest26814-21fschecks_count20:42:400
25548992316,5cyclictest26814-21fschecks_count20:42:400
25548992316,5cyclictest14678-21systemd-journal19:22:380
25548992316,5cyclictest14678-21systemd-journal19:22:380
25548992316,5cyclictest10710-21sshd21:25:240
25548992316,5cyclictest10710-21sshd21:25:240
25548992314,7cyclictest28509-21ls23:37:500
25548992314,7cyclictest28509-21ls23:37:500
25548992311,10cyclictest0-21swapper/020:32:410
25548992311,10cyclictest0-21swapper/020:32:410
2556399221,4cyclictest2477-21chrt22:28:502
2556399221,4cyclictest2477-21chrt22:28:502
2556399221,4cyclictest1863-21chrt23:52:522
2556399221,4cyclictest1863-21chrt23:52:522
2556399221,3cyclictest306682sleep223:44:162
2556399221,3cyclictest306682sleep223:44:162
2556399221,3cyclictest136882sleep222:58:042
2556399221,3cyclictest136882sleep222:58:042
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional