You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-05 - 07:36

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #4, slot #1

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1s.osadl.org (updated Mon Jan 05, 2026 00:44:11)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2697221070,3sleep026976-21cut20:30:370
1649421070,1chrt16495-21/usr/sbin/munin00:20:450
1649421070,1chrt16495-21/usr/sbin/munin00:20:450
1499021040,3chrt0-21swapper/322:50:393
200962840,3sleep20-21swapper/200:30:442
200962840,3sleep20-21swapper/200:30:442
4862780,3sleep20-21swapper/223:40:312
4862780,3sleep20-21swapper/223:40:312
174492740,22chrt0-21swapper/021:30:400
174492740,22chrt0-21swapper/021:30:400
164222720,2sleep016589-21grep20:00:470
164222720,2sleep016589-21grep20:00:470
190462680,2chrt0-21swapper/020:10:270
190462680,2chrt0-21swapper/020:10:270
51912670,3sleep20-21swapper/219:30:452
149392620,4chrt0-21swapper/222:50:382
71222600,2chrt6249-21/usr/sbin/munin23:55:450
71222600,2chrt6249-21/usr/sbin/munin23:55:450
71222600,2chrt6249-21/usr/sbin/munin23:55:450
241832600,3sleep00-21swapper/021:50:290
241832600,3sleep00-21swapper/021:50:290
185232580,2sleep10-21swapper/123:00:381
185232580,2sleep10-21swapper/123:00:381
185232580,2sleep10-21swapper/123:00:381
157102550,2sleep20-21swapper/200:20:352
156852540,4sleep20-21swapper/221:25:432
156852540,4sleep20-21swapper/221:25:432
227592510,2sleep00-21swapper/023:10:500
131532510,3sleep20-21swapper/221:20:362
43362480,3sleep20-21swapper/222:20:472
43362480,3sleep20-21swapper/222:20:472
57752470,2sleep10-21swapper/121:00:381
57752470,2sleep10-21swapper/121:00:381
275922410,8sleep0201rcuc/023:25:380
275922410,8sleep0201rcuc/023:25:380
117492400,3sleep20-21swapper/222:40:492
117492400,3sleep20-21swapper/222:40:492
34642380,2sleep03465-21gltestperf22:20:370
34642380,2sleep03465-21gltestperf22:20:370
94292370,3chrt0-21swapper/221:10:352
94292370,3chrt0-21swapper/221:10:352
305462370,3chrt0-21swapper/222:05:392
232852370,2sleep20-21swapper/220:20:382
232852370,2sleep20-21swapper/220:20:382
191122370,2sleep019118-21sh00:30:230
29342993630,4cyclictest579-21fschecks_count19:20:371
29342993630,4cyclictest579-21fschecks_count19:20:371
112692360,4sleep20-21swapper/200:05:482
91652350,2sleep016-21ksoftirqd/021:10:290
29341993529,4cyclictest572-21cups-browsed21:40:230
29341993529,4cyclictest572-21cups-browsed21:40:230
29341993529,4cyclictest572-21cups-browsed21:40:230
29341993528,5cyclictest6629-21latency_hist22:30:280
29341993528,5cyclictest6629-21latency_hist22:30:280
228702350,3sleep20-21swapper/221:45:382
228702350,3sleep20-21swapper/221:45:382
134662350,3sleep00-21swapper/021:20:380
29342993429,3cyclictest21559-21kworker/u9:3-xprtiod00:01:141
29342993427,5cyclictest20297-21sendmail21:40:231
29342993427,5cyclictest20297-21sendmail21:40:231
29342993427,5cyclictest20297-21sendmail21:40:231
29342993426,6cyclictest20045-21netstat00:30:421
29342993426,6cyclictest20045-21netstat00:30:421
29341993426,6cyclictest13552-21latency_hist00:15:240
29341993426,6cyclictest13552-21latency_hist00:15:240
2934199342,28cyclictest8631-21irqstats00:00:410
101222340,1sleep110123-21sshd22:37:271
101222340,1sleep110123-21sshd22:37:271
29341993326,6cyclictest1357-21netstat23:40:420
29341993326,6cyclictest1357-21netstat23:40:420
253742330,1sleep125399-21latency_hist23:20:251
253742330,1sleep125399-21latency_hist23:20:251
153012330,2sleep015404-21interrupts21:25:380
153012330,2sleep015404-21interrupts21:25:380
29342993228,3cyclictest20244-21gltestperf23:05:381
29342993228,3cyclictest20244-21gltestperf23:05:381
29342993227,3cyclictest0-21swapper/100:00:231
29342993227,3cyclictest0-21swapper/100:00:231
29342993227,3cyclictest0-21swapper/100:00:231
29341993225,5cyclictest10523-21netstat19:45:390
29341993225,5cyclictest10523-21netstat19:45:390
29341993224,6cyclictest4219-21latency_hist23:50:250
29341993224,6cyclictest4219-21latency_hist23:50:250
27192320,4sleep20-21swapper/220:50:452
27192320,4sleep20-21swapper/220:50:452
29342993124,5cyclictest10955-21gltestperf22:40:381
29342993124,5cyclictest10955-21gltestperf22:40:381
29342993123,6cyclictest7613-21fschecks_time21:05:411
29342993123,6cyclictest545-21in:imuxsock19:45:231
29342993123,6cyclictest545-21in:imuxsock19:45:231
29341993124,5cyclictest29340-21cyclictest20:56:230
29341993124,5cyclictest29340-21cyclictest20:56:230
29341993123,6cyclictest6275-21cut19:35:370
29341993123,6cyclictest6275-21cut19:35:370
2934499301,25cyclictest28947-21chrt23:29:023
2934499301,25cyclictest28947-21chrt23:29:023
2934499301,24cyclictest4057-21chrt23:49:043
2934499301,24cyclictest4057-21chrt23:49:043
29342993024,4cyclictest21650-21timerwakeupswit21:40:461
29342993024,4cyclictest21650-21timerwakeupswit21:40:461
29342993024,3cyclictest0-21swapper/122:20:361
29342993024,3cyclictest0-21swapper/122:20:361
29342993022,6cyclictest14429-21netstat00:15:421
29342993022,6cyclictest14429-21netstat00:15:421
29342993022,6cyclictest12005-21fschecks_count00:10:351
29342993022,6cyclictest12005-21fschecks_count00:10:351
29341993025,4cyclictest1346-21avahi-daemon00:30:460
29341993025,4cyclictest1346-21avahi-daemon00:30:460
29341993025,3cyclictest30140-21cron20:40:220
29341993025,3cyclictest30140-21cron20:40:220
29341993022,6cyclictest31445-21gltestperf23:35:360
29341993022,6cyclictest31445-21gltestperf23:35:360
29341993021,7cyclictest2174-21latency_hist19:25:240
29341993021,7cyclictest2174-21latency_hist19:25:240
29342992924,3cyclictest22669-21latency_hist20:20:231
29342992924,3cyclictest22669-21latency_hist20:20:231
29342992924,3cyclictest22669-21latency_hist20:20:231
29342992923,5cyclictest2244-21kernelversion20:50:381
29342992923,5cyclictest2244-21kernelversion20:50:381
29342992921,6cyclictest749-21runrttasks20:07:521
29342992921,6cyclictest749-21runrttasks20:07:521
29342992921,6cyclictest29340-21cyclictest20:30:431
29342992921,5cyclictest22978-21cpuspeed20:20:361
29342992921,5cyclictest22978-21cpuspeed20:20:361
29341992923,4cyclictest9044-21gltestperf22:35:370
29341992923,4cyclictest9044-21gltestperf22:35:370
29341992923,4cyclictest16539-21cut22:55:350
29341992923,4cyclictest16539-21cut22:55:350
29341992923,4cyclictest16539-21cut22:55:350
29341992922,5cyclictest32233-21kernelversion22:10:400
29341992922,5cyclictest32233-21kernelversion22:10:400
29341992922,5cyclictest29340-21cyclictest19:26:220
29341992922,5cyclictest14223-21needreboot19:55:420
29341992922,5cyclictest14223-21needreboot19:55:420
29341992921,6cyclictest4583-21cut23:50:350
29341992921,6cyclictest4583-21cut23:50:350
94292280,4sleep20-21swapper/200:00:532
2934499281,23cyclictest10141-21chrt22:37:403
2934499281,23cyclictest10141-21chrt22:37:403
2934499280,23cyclictest30060-21chrt20:39:213
2934499280,23cyclictest30060-21chrt20:39:213
2934499280,22cyclictest17541-21chrt21:30:433
2934499280,22cyclictest17541-21chrt21:30:433
29342992823,3cyclictest12623-21sensors_temp19:50:491
29342992823,3cyclictest12623-21sensors_temp19:50:491
29342992822,4cyclictest16321-21fschecks_count22:55:331
29342992822,4cyclictest16321-21fschecks_count22:55:331
29342992822,4cyclictest16321-21fschecks_count22:55:331
29342992822,4cyclictest0-21swapper/123:40:231
29342992822,4cyclictest0-21swapper/123:40:231
29342992821,5cyclictest2742-21fschecks_time23:45:361
29342992821,5cyclictest2742-21fschecks_time23:45:361
29341992822,5cyclictest21549-21threads21:40:460
29341992822,5cyclictest21549-21threads21:40:460
29341992822,4cyclictest29340-21cyclictest23:20:220
29341992822,4cyclictest29340-21cyclictest23:20:220
29341992822,4cyclictest26127-21/usr/sbin/munin21:55:400
29341992822,4cyclictest26127-21/usr/sbin/munin21:55:400
29341992822,4cyclictest26127-21/usr/sbin/munin21:55:400
29341992820,6cyclictest11958-21fschecks_time19:50:410
29341992820,6cyclictest11958-21fschecks_time19:50:410
2934499271,22cyclictest20622-21chrt20:13:123
2934499271,22cyclictest20622-21chrt20:13:123
2934499271,22cyclictest0-21swapper/323:35:383
2934499271,22cyclictest0-21swapper/323:35:383
2934499271,21cyclictest6839-21chrt23:55:403
2934499271,21cyclictest6839-21chrt23:55:403
2934499271,21cyclictest6839-21chrt23:55:403
2934499271,21cyclictest3834-21chrt19:28:253
2934499271,21cyclictest29384-21chrt22:01:543
2934499271,21cyclictest22626-21chrt20:19:553
2934499271,21cyclictest22626-21chrt20:19:553
2934499271,21cyclictest22626-21chrt20:19:553
2934499271,21cyclictest2002-21chrt19:23:513
2934499271,21cyclictest2002-21chrt19:23:513
2934499271,21cyclictest1416-21chrt20:49:103
2934499271,21cyclictest0-21swapper/319:47:323
2934499271,21cyclictest0-21swapper/319:47:323
2934499271,20cyclictest0-21swapper/321:18:533
2934499271,20cyclictest0-21swapper/321:18:533
2934499271,20cyclictest0-21swapper/300:27:293
2934499270,21cyclictest25006-21chrt23:16:403
2934499270,21cyclictest25006-21chrt23:16:403
2934499270,21cyclictest1985-21chrt23:41:243
2934499270,21cyclictest1985-21chrt23:41:243
29342992720,5cyclictest3987-21cron19:30:241
29342992720,5cyclictest26397-21fschecks_time21:55:371
29342992720,5cyclictest26397-21fschecks_time21:55:371
29342992720,5cyclictest26397-21fschecks_time21:55:371
29342992719,6cyclictest7711-21grep22:30:441
29342992719,6cyclictest7711-21grep22:30:441
29342992719,6cyclictest25783-21threads20:25:451
29341992722,3cyclictest0-21swapper/022:10:220
29341992721,5cyclictest21185-21cut20:15:380
29341992721,5cyclictest21185-21cut20:15:380
29341992721,5cyclictest21185-21cut20:15:380
29341992721,3cyclictest0-21swapper/022:45:370
29341992720,6cyclictest7353-21kernelversion22:30:380
29341992720,6cyclictest7353-21kernelversion22:30:380
29341992719,6cyclictest7337-21latency_hist21:05:300
29341992719,6cyclictest24772-21/usr/sbin/munin20:25:410
29341992719,6cyclictest1834-21missed_timers22:15:380
29341992719,6cyclictest1834-21missed_timers22:15:380
29341992719,6cyclictest1834-21missed_timers22:15:380
29341992718,7cyclictest30920-21cron23:35:240
29341992718,7cyclictest30920-21cron23:35:240
29341992718,6cyclictest15027-21netstat22:50:400
29341992715,3cyclictest90-1pr/ttyS020:52:540
29341992715,3cyclictest90-1pr/ttyS020:52:540
326222260,3sleep20-21swapper/220:45:352
2934499261,21cyclictest92542sleep319:40:593
2934499261,21cyclictest92542sleep319:40:593
2934499261,21cyclictest11887-21chrt22:40:543
2934499261,21cyclictest11887-21chrt22:40:543
2934499261,20cyclictest31231-21chrt22:06:453
2934499261,20cyclictest26110-21chrt20:26:343
2934499261,20cyclictest23807-21chrt21:47:273
2934499261,20cyclictest23807-21chrt21:47:273
2934499261,20cyclictest22797-21chrt00:39:503
2934499261,20cyclictest22797-21chrt00:39:503
2934499261,20cyclictest16744-21chrt20:01:133
2934499261,20cyclictest16744-21chrt20:01:133
2934499261,19cyclictest24235-21chrt20:21:243
2934499261,19cyclictest24235-21chrt20:21:243
2934499261,19cyclictest14918-21taskset19:56:593
2934499261,19cyclictest14918-21taskset19:56:593
2934499260,22cyclictest18698-21chrt20:07:253
2934499260,22cyclictest18698-21chrt20:07:253
2934499260,21cyclictest23101-21chrt23:11:053
29342992621,4cyclictest12793-21idleruntime-cro21:20:231
29342992621,4cyclictest12793-21idleruntime-cro21:20:231
29342992620,5cyclictest29778-21kernelversion23:30:371
29342992620,5cyclictest29778-21kernelversion23:30:371
29342992617,7cyclictest24589-21gltestperf21:50:371
29342992617,7cyclictest24589-21gltestperf21:50:371
29341992622,3cyclictest22382-21unixbench_multi00:35:480
29341992622,3cyclictest22382-21unixbench_multi00:35:480
29341992620,4cyclictest29340-21cyclictest19:40:480
29341992620,4cyclictest29340-21cyclictest19:40:480
29341992618,6cyclictest11019-21latency_hist21:15:270
29341992618,6cyclictest11019-21latency_hist21:15:270
29341992617,7cyclictest18775-21netstat23:00:420
29341992617,7cyclictest18775-21netstat23:00:420
29341992617,7cyclictest18775-21netstat23:00:420
286002260,3sleep20-21swapper/220:35:342
286002260,3sleep20-21swapper/220:35:342
2934499251,20cyclictest7408-21chrt19:36:293
2934499251,20cyclictest7408-21chrt19:36:293
2934499251,20cyclictest6449-21chrt22:28:073
2934499251,20cyclictest6449-21chrt22:28:073
2934499251,20cyclictest5755-21chrt23:51:583
2934499251,20cyclictest5755-21chrt23:51:583
2934499251,20cyclictest5521-21chrt19:31:103
2934499251,20cyclictest4919-21chrt20:56:133
2934499251,20cyclictest4919-21chrt20:56:133
2934499251,20cyclictest3144-21chrt20:52:193
2934499251,20cyclictest3144-21chrt20:52:193
2934499251,20cyclictest24922-21chrt21:50:413
2934499251,20cyclictest24922-21chrt21:50:413
2934499251,20cyclictest17625-21chrt22:57:503
2934499251,20cyclictest17625-21chrt22:57:503
2934499251,20cyclictest17625-21chrt22:57:503
2934499251,20cyclictest13837-21chrt22:47:013
2934499251,20cyclictest12111-21chrt00:10:353
2934499251,20cyclictest12111-21chrt00:10:353
2934499251,19cyclictest8725-21taskset21:06:593
2934499251,19cyclictest27456-21chrt21:56:283
2934499251,19cyclictest27456-21chrt21:56:283
2934499251,19cyclictest27456-21chrt21:56:283
2934499251,19cyclictest19659-21chrt23:04:463
2934499251,19cyclictest19659-21chrt23:04:463
2934499251,19cyclictest19659-21chrt23:04:463
2934499251,19cyclictest16940-21chrt00:21:213
2934499251,19cyclictest16940-21chrt00:21:213
2934499251,19cyclictest0-21swapper/323:21:263
2934499250,21cyclictest19932-21chrt21:35:493
2934499250,21cyclictest19932-21chrt21:35:493
2934499250,21cyclictest19932-21chrt21:35:493
2934499250,20cyclictest6846-21chrt21:01:483
2934499250,20cyclictest6846-21chrt21:01:483
2934499250,20cyclictest30745-21chrt23:33:083
2934499250,20cyclictest30745-21chrt23:33:083
2934399251,20cyclictest8016-21chrt00:00:292
2934399251,20cyclictest8016-21chrt00:00:292
2934399251,20cyclictest8016-21chrt00:00:292
29342992519,4cyclictest30865-21switchtime22:05:451
29342992517,6cyclictest16593-21unixbench-2d20:00:471
29342992517,6cyclictest16593-21unixbench-2d20:00:471
29341992519,4cyclictest29340-21cyclictest22:43:000
29341992519,4cyclictest29340-21cyclictest22:43:000
2934499241,19cyclictest969-21taskset22:15:013
2934499241,19cyclictest969-21taskset22:15:013
2934499241,19cyclictest8385-21taskset22:34:023
2934499241,19cyclictest8385-21taskset22:34:023
2934499241,19cyclictest4854-21chrt22:25:263
2934499241,19cyclictest4854-21chrt22:25:263
2934499241,19cyclictest2515-21chrt22:15:493
2934499241,19cyclictest2515-21chrt22:15:493
2934499241,19cyclictest2515-21chrt22:15:493
2934499241,19cyclictest22000-21chrt21:43:093
2934499241,19cyclictest22000-21chrt21:43:093
2934499241,19cyclictest20638-21taskset00:31:013
2934499241,19cyclictest20638-21taskset00:31:013
2934499241,19cyclictest16437-21chrt21:28:463
2934499241,19cyclictest16437-21chrt21:28:463
2934499241,19cyclictest13055-21chrt19:52:013
2934499241,19cyclictest13055-21chrt19:52:013
2934499241,18cyclictest8902-21taskset00:00:463
2934499241,18cyclictest10670-21chrt00:05:413
2934499240,20cyclictest27921-21chrt20:30:523
2934499240,19cyclictest21312-21chrt23:07:033
2934499240,19cyclictest21312-21chrt23:07:033
2934499240,19cyclictest15040-21chrt00:16:143
2934499240,19cyclictest15040-21chrt00:16:143
29342992420,3cyclictest27769-21latency_hist22:00:371
29342992420,2cyclictest0-21swapper/121:30:361
29342992420,2cyclictest0-21swapper/121:30:361
29342992419,3cyclictest28729-21fschecks_time20:35:351
29342992419,3cyclictest28729-21fschecks_time20:35:351
29342992419,3cyclictest0-21swapper/123:40:371
29342992419,3cyclictest0-21swapper/123:40:371
29342992416,6cyclictest5170-21fschecks_count22:25:361
29342992416,6cyclictest5170-21fschecks_count22:25:361
29341992420,3cyclictest29340-21cyclictest00:15:390
29341992420,3cyclictest29340-21cyclictest00:15:390
29341992420,2cyclictest0-21swapper/023:20:460
29341992420,2cyclictest0-21swapper/023:05:430
29341992420,2cyclictest0-21swapper/023:05:430
29341992419,3cyclictest0-21swapper/021:50:480
29341992419,3cyclictest0-21swapper/021:50:480
29341992419,3cyclictest0-21swapper/019:30:450
2934499231,18cyclictest14444-21chrt21:22:433
2934499231,17cyclictest31293-21chrt20:40:453
29342992319,2cyclictest0-21swapper/100:20:451
29342992319,2cyclictest0-21swapper/100:20:451
29342992318,3cyclictest0-21swapper/123:25:491
29342992318,3cyclictest0-21swapper/123:25:491
29342992318,3cyclictest0-21swapper/119:55:431
29342992318,3cyclictest0-21swapper/119:55:431
29342992317,4cyclictest0-21swapper/120:45:341
29342992317,4cyclictest0-21swapper/120:10:471
29342992317,4cyclictest0-21swapper/120:10:471
29342992317,3cyclictest0-21swapper/122:50:351
29342992316,5cyclictest2916-21sendmail22:20:231
29342992316,5cyclictest2916-21sendmail22:20:231
29342992316,5cyclictest2916-21sendmail22:20:231
29342992316,5cyclictest17657-21fschecks_count00:25:371
29342992315,6cyclictest23410-21threads21:45:451
29342992315,6cyclictest23410-21threads21:45:451
29341992319,2cyclictest0-21swapper/021:15:400
29341992319,2cyclictest0-21swapper/021:15:400
29341992318,3cyclictest0-21swapper/022:00:430
29341992318,3cyclictest0-21swapper/020:45:430
29341992318,3cyclictest0-21swapper/020:10:460
29341992318,3cyclictest0-21swapper/020:10:460
29341992317,4cyclictest23992-21timerwakeupswit20:20:470
29341992317,4cyclictest23992-21timerwakeupswit20:20:470
21322230,2sleep20-21swapper/219:25:222
21322230,2sleep20-21swapper/219:25:222
2934499221,16cyclictest9624-21chrt21:10:363
2934499221,16cyclictest9624-21chrt21:10:363
29342992217,4cyclictest32273-21latency22:10:401
29342992217,4cyclictest32273-21latency22:10:401
29342992217,3cyclictest0-21swapper/121:25:221
29342992217,3cyclictest0-21swapper/121:10:401
29342992217,3cyclictest0-21swapper/121:10:401
29342992217,3cyclictest0-21swapper/120:55:421
29342992217,3cyclictest0-21swapper/120:55:421
29342992217,3cyclictest0-21swapper/100:40:231
29342992217,3cyclictest0-21swapper/100:40:231
29342992215,5cyclictest5338-21switchtime23:50:451
29342992215,5cyclictest5338-21switchtime23:50:451
29342992214,6cyclictest16191-21users21:25:471
29342992214,6cyclictest16191-21users21:25:471
29342992213,7cyclictest13972-21seq22:48:471
29342992116,3cyclictest0-21swapper/119:30:341
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional