You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-12 - 07:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #4, slot #1

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1s.osadl.org (updated Tue May 12, 2026 00:44:14)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
553521100,1sleep10-21swapper/121:37:381
124262950,5sleep012429-21head20:27:460
124262950,5sleep012429-21head20:27:460
306192940,2chrt30618-21awk22:42:450
306192940,2chrt30618-21awk22:42:450
306192940,2chrt30618-21awk22:42:450
256372820,3chrt0-21swapper/023:57:390
256372820,3chrt0-21swapper/023:57:390
318572780,3sleep131858-21sshd19:53:581
318572780,3sleep131858-21sshd19:53:581
255312620,3sleep30-21swapper/323:57:393
255312620,3sleep30-21swapper/323:57:393
110672590,2chrt281ktimers/120:27:271
110672590,2chrt281ktimers/120:27:271
110672590,2chrt281ktimers/120:27:271
211122580,2sleep00-21swapper/020:52:400
211122580,2sleep00-21swapper/020:52:400
187542570,2sleep118755-21cut20:47:331
38142530,3sleep10-21swapper/121:32:391
293652530,3sleep10-21swapper/122:42:301
293652530,3sleep10-21swapper/122:42:301
178552500,3sleep20-21swapper/222:11:002
178552500,3sleep20-21swapper/222:11:002
76172460,2sleep30-21swapper/321:42:423
76172460,2sleep30-21swapper/321:42:423
76172460,2sleep30-21swapper/321:42:423
216152450,3chrt0-21swapper/320:52:463
216152450,3chrt0-21swapper/320:52:463
98332440,3sleep30-21swapper/321:47:463
98332440,3sleep30-21swapper/321:47:463
271802440,3sleep20-21swapper/200:02:352
271802440,3sleep20-21swapper/200:02:352
271802440,3sleep20-21swapper/200:02:352
319772430,4sleep20-21swapper/221:22:352
319772430,4sleep20-21swapper/221:22:352
175772430,3sleep30-21swapper/320:42:423
175772430,3sleep30-21swapper/320:42:423
262582420,2sleep20-21swapper/222:32:382
262582420,2sleep20-21swapper/222:32:382
183022410,2sleep118301-21df_inode22:12:351
183022410,2sleep118301-21df_inode22:12:351
1566599393,29cyclictest27372-21/usr/sbin/munin00:02:360
1566599393,29cyclictest27372-21/usr/sbin/munin00:02:360
1566599393,29cyclictest27372-21/usr/sbin/munin00:02:360
200162350,4sleep30-21swapper/319:22:413
15666993428,4cyclictest23598-21latency_hist22:27:231
15666993428,4cyclictest23598-21latency_hist22:27:231
15666993425,7cyclictest0-21swapper/120:17:461
15666993425,7cyclictest0-21swapper/120:17:461
15665993427,5cyclictest27322-21idleruntime-cro22:37:230
15665993427,5cyclictest27322-21idleruntime-cro22:37:230
127432340,1sleep3431rcuc/323:22:403
127432340,1sleep3431rcuc/323:22:403
15666993327,4cyclictest17986-21latency_hist22:12:221
15666993327,4cyclictest17986-21latency_hist22:12:221
15666993325,6cyclictest8684-21latency_hist21:47:281
15666993325,6cyclictest8684-21latency_hist21:47:281
15666993325,6cyclictest8684-21latency_hist21:47:281
15666993325,6cyclictest26813-21latency_hist00:02:251
15666993325,6cyclictest26813-21latency_hist00:02:251
15666993325,6cyclictest16453-21netstat23:32:411
15666993324,7cyclictest1796-21kernelversion21:27:381
15666993324,7cyclictest1796-21kernelversion21:27:381
15666993324,7cyclictest1796-21kernelversion21:27:381
1566699331,29cyclictest27519-21sed00:02:381
1566699331,29cyclictest27519-21sed00:02:381
1566699331,29cyclictest27519-21sed00:02:381
31382320,2sleep115664-21cyclictest22:57:381
31382320,2sleep115664-21cyclictest22:57:381
274192320,3sleep30-21swapper/319:42:393
274192320,3sleep30-21swapper/319:42:393
15666993226,4cyclictest2402-21latency_hist22:57:241
15666993226,4cyclictest2402-21latency_hist22:57:241
15666993127,3cyclictest27863-21cron21:12:221
15666993127,3cyclictest27863-21cron21:12:221
15666993124,5cyclictest547-21rs:main1
15666993124,5cyclictest547-21rs:main1
15666993124,5cyclictest547-21rs:main1
15666993124,5cyclictest22371-21threads23:47:461
15666993124,5cyclictest22371-21threads23:47:461
15666993122,7cyclictest28346-21cut21:12:361
187822300,3sleep018787-21unixbench_multi23:37:450
1566899301,22cyclictest26842-21taskset00:02:272
1566899301,22cyclictest26842-21taskset00:02:272
15666993023,5cyclictest6015-21sshd23:06:001
15666993023,5cyclictest6015-21sshd23:06:001
15666993023,5cyclictest6015-21sshd23:06:001
15666993023,5cyclictest11710-21sshd23:22:001
15666993023,5cyclictest11710-21sshd23:22:001
15666993023,5cyclictest0-21swapper/120:52:231
15666993023,5cyclictest0-21swapper/120:52:231
15666993021,7cyclictest25606-21latency_hist22:32:311
15666993021,7cyclictest25606-21latency_hist22:32:311
15665993024,5cyclictest28396-21cron19:47:220
15665993024,5cyclictest28396-21cron19:47:220
15665993023,5cyclictest29670-21timerandwakeup19:47:460
15665993023,5cyclictest29670-21timerandwakeup19:47:460
15665993023,5cyclictest11756-21latency_hist23:22:230
15665993023,5cyclictest11756-21latency_hist23:22:230
15665993016,12cyclictest0-21swapper/021:22:360
15665993016,12cyclictest0-21swapper/021:22:360
15665993015,13cyclictest0-21swapper/022:02:370
15665993015,13cyclictest0-21swapper/022:02:370
15666992922,5cyclictest22422-21latency_hist20:57:291
15666992922,5cyclictest22422-21latency_hist20:57:291
15666992922,5cyclictest21134-21latency_hist19:27:331
15665992919,8cyclictest1-21systemd23:56:000
15665992919,8cyclictest1-21systemd23:56:000
194182280,2sleep30-21swapper/323:42:303
1566899281,8cyclictest22639-21chrt23:47:482
1566899281,8cyclictest22639-21chrt23:47:482
15666992823,4cyclictest14678-21systemd-journal00:10:001
15666992823,4cyclictest14678-21systemd-journal00:10:001
15666992822,4cyclictest0-21swapper/120:12:461
15666992822,4cyclictest0-21swapper/120:12:461
15666992821,5cyclictest28868-21fschecks_time19:47:361
15666992821,5cyclictest28868-21fschecks_time19:47:361
15666992821,5cyclictest13627-21kernelversion20:32:371
15666992821,5cyclictest13627-21kernelversion20:32:371
15665992821,5cyclictest19704-21fschecks_time23:42:360
15665992821,5cyclictest19704-21fschecks_time23:42:360
15665992821,5cyclictest19704-21fschecks_time23:42:360
301752270,3sleep129969-21/usr/sbin/munin21:17:361
301752270,3sleep129969-21/usr/sbin/munin21:17:361
1566899271,9cyclictest4646-21taskset00:27:442
1566899271,9cyclictest4646-21taskset00:27:442
1566899271,9cyclictest4646-21taskset00:27:442
15666992722,4cyclictest9037-21fschecks_time21:47:361
15666992722,4cyclictest9037-21fschecks_time21:47:361
15666992722,3cyclictest0-21swapper/122:02:231
15666992722,3cyclictest0-21swapper/122:02:231
15666992721,5cyclictest23720-21kernelversion23:52:381
15666992721,5cyclictest23720-21kernelversion23:52:381
15666992721,3cyclictest0-21swapper/123:37:451
15666992720,5cyclictest15243-21cut20:37:361
15666992720,5cyclictest15243-21cut20:37:361
15666992720,5cyclictest1167-21latency_hist21:27:281
15666992720,5cyclictest1167-21latency_hist21:27:281
15666992719,6cyclictest8770-21ls23:12:381
15666992719,6cyclictest8770-21ls23:12:381
15666992719,6cyclictest16265-21latency_hist22:07:311
15666992719,6cyclictest16265-21latency_hist22:07:311
15665992722,3cyclictest5382-21head21:37:370
15665992722,3cyclictest5382-21head21:37:370
15665992722,3cyclictest0-21swapper/000:22:230
15665992722,3cyclictest0-21swapper/000:22:230
15665992721,4cyclictest7785-21gltestperf20:17:380
15665992721,4cyclictest7785-21gltestperf20:17:380
15665992721,4cyclictest21890-21ntp_offset19:27:410
15665992720,5cyclictest2238-21fschecks_time00:22:360
15665992720,5cyclictest2238-21fschecks_time00:22:360
15665992717,8cyclictest0-21swapper/022:57:440
15665992717,8cyclictest0-21swapper/022:57:440
1566899268,3cyclictest276272chrt21:09:212
1566899268,3cyclictest276272chrt21:09:212
15666992621,3cyclictest0-21swapper/119:57:461
15666992620,5cyclictest6516-21ntp_offset00:32:421
15666992620,5cyclictest6516-21ntp_offset00:32:421
15666992620,3cyclictest0-21swapper/119:42:441
15666992620,3cyclictest0-21swapper/119:42:441
15666992619,5cyclictest21864-21latency_hist22:22:301
15666992619,5cyclictest21864-21latency_hist22:22:301
15666992619,5cyclictest21864-21latency_hist22:22:301
15666992618,6cyclictest24645-21cut21:02:361
15666992618,6cyclictest24645-21cut21:02:361
15666992618,6cyclictest24645-21cut21:02:361
15665992622,2cyclictest0-21swapper/019:42:230
15665992622,2cyclictest0-21swapper/019:42:230
15665992621,3cyclictest0-21swapper/000:32:460
15665992621,3cyclictest0-21swapper/000:32:460
15665992620,5cyclictest17101-21fschecks_time20:42:350
15665992620,5cyclictest17101-21fschecks_time20:42:350
15665992620,5cyclictest16477-21ntp_kernel_pll_23:32:410
15665992620,4cyclictest32475-21unixbench_multi22:47:460
15665992620,4cyclictest32475-21unixbench_multi22:47:460
15665992619,5cyclictest11767-21timerwakeupswit21:52:430
15665992618,6cyclictest1432-21fschecks_count21:27:360
15665992618,6cyclictest1432-21fschecks_count21:27:360
15665992618,6cyclictest1432-21fschecks_count21:27:360
15665992617,7cyclictest30454-21irqstats21:17:380
15665992617,7cyclictest30454-21irqstats21:17:380
15665992613,11cyclictest14678-21systemd-journal21:43:190
15665992613,11cyclictest14678-21systemd-journal21:43:190
15665992613,11cyclictest14678-21systemd-journal21:43:190
73322250,4sleep30-21swapper/320:17:283
73322250,4sleep30-21swapper/320:17:283
1566899252,7cyclictest0-21swapper/219:57:282
1566899252,7cyclictest0-21swapper/219:57:282
15666992519,5cyclictest1944-21fschecks_count20:02:351
15666992519,5cyclictest1944-21fschecks_count20:02:351
15666992519,5cyclictest1944-21fschecks_count20:02:351
15666992519,4cyclictest0-21swapper/122:42:381
15666992519,4cyclictest0-21swapper/122:42:381
15666992519,4cyclictest0-21swapper/122:42:381
15666992519,4cyclictest0-21swapper/100:17:451
15666992519,4cyclictest0-21swapper/100:17:451
15666992519,3cyclictest0-21swapper/120:42:461
15666992518,5cyclictest8114-21latency_hist23:12:291
15666992518,5cyclictest8114-21latency_hist23:12:291
15666992517,6cyclictest7643-21/usr/sbin/munin00:37:411
15666992517,6cyclictest7643-21/usr/sbin/munin00:37:411
15666992517,6cyclictest31400-21netstat00:12:401
15666992517,6cyclictest31400-21netstat00:12:401
15665992521,3cyclictest26568-21cut21:07:370
15665992521,3cyclictest26568-21cut21:07:370
15665992521,2cyclictest0-21swapper/000:12:230
15665992521,2cyclictest0-21swapper/000:12:230
80162240,2chrt0-21swapper/320:17:403
80162240,2chrt0-21swapper/320:17:403
74652240,3sleep20-21swapper/220:17:352
74652240,3sleep20-21swapper/220:17:352
1567499245,11cyclictest0-21swapper/300:02:363
1567499245,11cyclictest0-21swapper/300:02:363
1567499245,11cyclictest0-21swapper/300:02:363
15674992417,4cyclictest147722sleep320:37:233
15674992417,4cyclictest147722sleep320:37:233
15668992418,4cyclictest184372chrt22:12:362
15668992418,4cyclictest184372chrt22:12:362
15666992420,2cyclictest0-21swapper/123:42:441
15666992420,2cyclictest0-21swapper/123:42:441
15666992420,2cyclictest0-21swapper/123:42:441
15666992419,3cyclictest0-21swapper/119:27:371
15666992418,3cyclictest0-21swapper/120:07:411
15666992418,3cyclictest0-21swapper/120:07:411
15665992420,2cyclictest0-21swapper/021:07:220
15665992420,2cyclictest0-21swapper/021:07:220
15665992420,2cyclictest0-21swapper/021:07:220
15665992419,3cyclictest0-21swapper/020:37:410
15665992419,3cyclictest0-21swapper/020:37:410
15665992418,5cyclictest28978-21sshd22:40:000
15665992418,5cyclictest28978-21sshd22:40:000
15665992418,4cyclictest0-21swapper/023:47:470
15665992418,4cyclictest0-21swapper/023:47:470
15665992418,4cyclictest0-21swapper/022:12:450
15665992418,4cyclictest0-21swapper/022:12:450
15665992417,5cyclictest6366-21processes20:12:430
15665992417,5cyclictest6366-21processes20:12:430
15665992417,5cyclictest4623-21ntp_kernel_pll_00:27:430
15665992417,5cyclictest4623-21ntp_kernel_pll_00:27:430
15665992417,5cyclictest4623-21ntp_kernel_pll_00:27:430
15665992416,6cyclictest8386-21ntp_states00:37:420
15665992416,6cyclictest8386-21ntp_states00:37:420
15665992415,7cyclictest9691-21gltestperf20:22:360
15665992415,7cyclictest9691-21gltestperf20:22:360
15665992415,7cyclictest9691-21gltestperf20:22:360
15665992415,7cyclictest1025-21grep19:57:460
71912230,2chrt0-21swapper/221:42:372
71912230,2chrt0-21swapper/221:42:372
71912230,2chrt0-21swapper/221:42:372
15666992319,2cyclictest0-21swapper/120:57:431
15666992319,2cyclictest0-21swapper/120:57:431
15666992318,4cyclictest26476-21processes22:32:431
15666992318,4cyclictest26476-21processes22:32:431
15666992317,5cyclictest2531-21kernelversion00:22:391
15666992317,5cyclictest2531-21kernelversion00:22:391
15666992317,5cyclictest11206-21latency21:52:361
15666992317,4cyclictest0-21swapper/120:32:291
15666992317,4cyclictest0-21swapper/120:32:291
15666992315,6cyclictest4882-21ls00:27:461
15666992315,6cyclictest4882-21ls00:27:461
15666992315,6cyclictest4882-21ls00:27:461
15666992315,6cyclictest447-21cron22:52:221
15666992315,6cyclictest447-21cron22:52:221
15665992319,2cyclictest0-21swapper/023:10:000
15665992319,2cyclictest0-21swapper/023:10:000
15665992318,3cyclictest0-21swapper/022:27:460
15665992318,3cyclictest0-21swapper/022:27:460
15665992318,3cyclictest0-21swapper/021:00:580
15665992318,3cyclictest0-21swapper/021:00:580
15665992317,4cyclictest0-21swapper/021:57:430
15665992317,4cyclictest0-21swapper/021:57:430
15665992317,4cyclictest0-21swapper/021:17:230
15665992317,4cyclictest0-21swapper/020:07:440
15665992317,4cyclictest0-21swapper/020:07:440
15665992316,5cyclictest12477-21kernelversion23:22:360
15665992316,5cyclictest12477-21kernelversion23:22:360
15666992217,3cyclictest0-21swapper/123:27:451
15666992217,3cyclictest0-21swapper/123:27:451
15665992218,2cyclictest0-21swapper/021:37:440
15665992218,2cyclictest0-21swapper/021:37:440
15665992218,2cyclictest0-21swapper/021:37:440
15665992218,2cyclictest0-21swapper/019:52:360
15665992218,2cyclictest0-21swapper/019:52:360
15665992216,5cyclictest1944-21fschecks_count20:02:350
15665992216,5cyclictest1944-21fschecks_count20:02:350
15665992216,5cyclictest1944-21fschecks_count20:02:350
15665992215,5cyclictest6577-21fschecks_time23:07:350
15665992215,5cyclictest0-21swapper/022:52:480
15665992215,5cyclictest0-21swapper/022:52:480
1566599221,11cyclictest20042-21ntp_states19:22:420
307192210,3sleep30-21swapper/321:17:423
307192210,3sleep30-21swapper/321:17:423
15666992114,5cyclictest25392-21kernelversion19:37:381
15666992114,5cyclictest25392-21kernelversion19:37:381
15665992116,3cyclictest0-21swapper/023:12:440
15665992116,3cyclictest0-21swapper/023:12:440
15665992116,3cyclictest0-21swapper/021:47:360
15665992116,3cyclictest0-21swapper/021:47:360
15665992115,4cyclictest572-21cups-browsed22:22:230
15665992115,4cyclictest572-21cups-browsed22:22:230
15665992115,4cyclictest572-21cups-browsed22:22:230
15665992115,4cyclictest0-21swapper/020:32:350
15665992115,4cyclictest0-21swapper/020:32:350
15665992114,5cyclictest23370-21tail19:32:350
15665992114,5cyclictest23370-21tail19:32:350
15665992114,5cyclictest0-21swapper/022:07:430
15665992114,5cyclictest0-21swapper/022:07:430
1566599211,15cyclictest5120-21head23:02:400
1566599211,15cyclictest5120-21head23:02:400
15666992015,3cyclictest0-21swapper/119:32:381
15666992015,3cyclictest0-21swapper/119:32:381
15665992015,3cyclictest14600-21grep23:27:410
15665992015,3cyclictest14600-21grep23:27:410
15665992015,3cyclictest0-21swapper/020:47:450
15665992015,3cyclictest0-21swapper/020:47:450
15665992015,3cyclictest0-21swapper/000:12:430
15665992015,3cyclictest0-21swapper/000:12:430
195122190,2sleep20-21swapper/220:47:422
195122190,2sleep20-21swapper/220:47:422
1567499191,13cyclictest895-21chrt22:52:373
1567499191,13cyclictest895-21chrt22:52:373
1566899192,3cyclictest0-21swapper/221:17:362
1566899192,3cyclictest0-21swapper/221:17:362
1566899191,13cyclictest16044-21chrt23:32:362
15666991915,2cyclictest0-21swapper/123:22:361
15666991915,2cyclictest0-21swapper/123:22:361
15665991914,3cyclictest0-21swapper/022:27:270
15665991914,3cyclictest0-21swapper/022:27:270
1566899181,13cyclictest28597-21taskset21:12:392
1566899181,13cyclictest11479-21taskset20:27:362
1566899181,13cyclictest11479-21taskset20:27:362
1566899181,12cyclictest12475-21chrt21:57:292
1566899181,12cyclictest10991-21taskset21:52:342
1566899181,12cyclictest10991-21taskset21:52:342
1566899181,12cyclictest10061-21taskset20:22:422
1566899181,12cyclictest10061-21taskset20:22:422
1566899181,12cyclictest10061-21taskset20:22:422
1567499171,9cyclictest0-21swapper/323:08:003
1567499171,9cyclictest0-21swapper/323:08:003
1567499171,11cyclictest8865-21chrt23:12:393
1567499171,11cyclictest8865-21chrt23:12:393
1567499171,11cyclictest4027-21chrt00:27:373
1567499171,11cyclictest4027-21chrt00:27:373
1567499171,11cyclictest4027-21chrt00:27:373
1567499171,11cyclictest23415-21chrt20:57:443
1567499171,11cyclictest23415-21chrt20:57:443
1567499171,11cyclictest21777-21chrt19:27:393
1567499171,11cyclictest15305-21chrt22:02:443
1567499171,11cyclictest15305-21chrt22:02:443
1567499171,11cyclictest13098-21taskset21:57:383
1567499171,11cyclictest13098-21taskset21:57:383
1567499171,11cyclictest11775-21taskset21:52:433
1567499171,11cyclictest0-21swapper/322:27:333
1567499171,11cyclictest0-21swapper/322:27:333
1566899171,9cyclictest0-21swapper/221:32:342
1566899171,9cyclictest0-21swapper/221:32:342
1566899171,9cyclictest0-21swapper/221:32:342
1566899171,12cyclictest30577-21chrt00:12:282
1566899171,12cyclictest30577-21chrt00:12:282
1566899171,12cyclictest25628-21chrt21:02:462
1566899171,12cyclictest25628-21chrt21:02:462
1566899171,12cyclictest25628-21chrt21:02:462
1566899171,11cyclictest22552-21taskset22:22:372
1566899171,11cyclictest22552-21taskset22:22:372
1567499162,10cyclictest0-21swapper/322:12:463
1567499162,10cyclictest0-21swapper/322:12:463
1567499161,11cyclictest4415-21chrt23:02:303
1567499161,11cyclictest4415-21chrt23:02:303
1567499161,11cyclictest21040-21chrt22:17:443
1567499161,11cyclictest21040-21chrt22:17:443
1567499161,11cyclictest21040-21chrt22:17:443
1567499161,11cyclictest0-21swapper/322:37:363
1567499161,11cyclictest0-21swapper/322:37:363
1567499161,11cyclictest0-21swapper/322:37:273
1567499161,11cyclictest0-21swapper/322:37:273
1567499161,11cyclictest0-21swapper/321:37:343
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional