You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-31 - 12:38

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #4, slot #1

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1s.osadl.org (updated Wed Dec 31, 2025 00:44:11)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
77502910,3sleep30-21swapper/321:45:353
77502910,3sleep30-21swapper/321:45:353
254762860,3sleep30-21swapper/321:05:393
215382780,3chrt0-21swapper/219:30:302
13432780,3chrt1345-21threads20:00:401
13432780,3chrt1345-21threads20:00:401
316342680,2sleep20-21swapper/200:15:372
316342680,2sleep20-21swapper/200:15:372
316342680,2sleep20-21swapper/200:15:372
283032660,2sleep2351rcuc/222:40:332
283032660,2sleep2351rcuc/222:40:332
63492650,2chrt0-21swapper/221:40:392
63492650,2chrt0-21swapper/221:40:392
235002590,3sleep30-21swapper/321:00:363
14107995852,3cyclictest0-21swapper/000:00:260
14107995852,3cyclictest0-21swapper/000:00:260
14107995852,3cyclictest0-21swapper/000:00:260
127522580,3chrt0-21swapper/120:30:431
280972550,3sleep20-21swapper/200:05:402
197182540,3sleep30-21swapper/322:15:423
197182540,3sleep30-21swapper/322:15:423
307102530,2sleep30-21swapper/322:45:413
307102530,2sleep30-21swapper/322:45:413
182382490,3chrt0-21swapper/319:20:343
182382490,3chrt0-21swapper/319:20:343
266642470,1sleep00-21swapper/000:00:480
266642470,1sleep00-21swapper/000:00:480
1411099471,30cyclictest28950-21latency_hist19:50:261
1411099464,32cyclictest25969-21memory00:00:361
1411099464,32cyclictest25969-21memory00:00:361
8922450,3sleep00-21swapper/000:20:340
8922450,3sleep00-21swapper/000:20:340
269792450,2sleep30-21swapper/319:45:223
269792450,2sleep30-21swapper/319:45:223
1411099451,31cyclictest8075-21taskset23:12:231
1411099451,31cyclictest8075-21taskset23:12:231
217592440,3sleep00-21swapper/019:30:320
202482430,3sleep20-21swapper/223:45:342
202482430,3sleep20-21swapper/223:45:342
1411099421,28cyclictest15753-21taskset23:34:501
1411099421,28cyclictest15753-21taskset23:34:501
1411099420,29cyclictest2939-21/usr/sbin/munin21:30:431
1411099420,29cyclictest2939-21/usr/sbin/munin21:30:431
103892420,3sleep30-21swapper/320:25:343
326062410,1sleep1346-21unixbench_singl22:50:421
326062410,1sleep1346-21unixbench_singl22:50:421
278412410,2sleep127844-21missed_timers00:05:351
272612410,3sleep00-21swapper/000:05:300
148632410,1sleep10-21swapper/122:05:311
148632410,1sleep10-21swapper/122:05:311
1411099411,28cyclictest31094-21seq22:47:151
1411099411,28cyclictest31094-21seq22:47:151
1411099410,25cyclictest21956-21taskset22:24:421
1411099410,25cyclictest21956-21taskset22:24:421
281512400,2sleep128154-21grep22:40:321
281512400,2sleep128154-21grep22:40:321
1411099400,38cyclictest1390-21sshd21:29:491
1411099400,38cyclictest1390-21sshd21:29:491
298622390,2sleep20-21swapper/219:50:402
14110993918,18cyclictest14102-21cyclictest00:20:441
14110993918,18cyclictest14102-21cyclictest00:20:441
1411099391,26cyclictest29899-21seq21:18:141
1411099391,26cyclictest29899-21seq21:18:141
1411099391,26cyclictest29899-21seq21:18:141
1411099391,25cyclictest30712-21runrttasks00:14:591
1411099391,25cyclictest30712-21runrttasks00:14:591
1411099391,24cyclictest0-21swapper/119:22:351
1411099391,24cyclictest0-21swapper/119:22:351
1411099390,37cyclictest25189-40mandb00:00:321
1411099390,37cyclictest25189-40mandb00:00:321
74732380,1sleep00-21swapper/023:10:390
74732380,1sleep00-21swapper/023:10:390
1411099381,35cyclictest0-21swapper/121:01:591
1411099381,34cyclictest4857-21ntp_states20:10:371
1411099381,34cyclictest4857-21ntp_states20:10:371
1411099381,34cyclictest4857-21ntp_states20:10:371
1411099381,34cyclictest31572-21ntp_states19:55:391
1411099381,34cyclictest31572-21ntp_states19:55:391
1411099381,34cyclictest31572-21ntp_states19:55:391
1411499371,25cyclictest26689-21taskset00:01:073
1411499371,25cyclictest26689-21taskset00:01:073
1411099371,34cyclictest601-21snmpd21:08:301
1411099371,33cyclictest28152-21sshd21:14:491
1411099371,33cyclictest22378-21sshd20:57:491
1411099371,33cyclictest22378-21sshd20:57:491
1411099370,34cyclictest23010-21sshd19:33:481
163492360,3sleep20-21swapper/222:09:502
163492360,3sleep20-21swapper/222:09:502
1411399363,23cyclictest25271-21chrt00:00:262
1411399363,23cyclictest25271-21chrt00:00:262
1411399363,23cyclictest25271-21chrt00:00:262
1411099369,25cyclictest729-21dbus-daemon23:50:391
1411099366,28cyclictest16616-21vmstat20:40:431
1411099366,28cyclictest16616-21vmstat20:40:431
14110993630,4cyclictest10173-21cron23:20:181
14110993630,4cyclictest10173-21cron23:20:181
1411099361,33cyclictest16768-21ntp_states23:35:381
1411099361,33cyclictest16768-21ntp_states23:35:381
1411099361,33cyclictest0-21swapper/120:23:581
1411099361,32cyclictest3491-21sshd20:05:491
1411099361,32cyclictest3491-21sshd20:05:491
14107993627,7cyclictest14102-21cyclictest19:55:400
14107993627,7cyclictest14102-21cyclictest19:55:400
14107993627,7cyclictest14102-21cyclictest19:55:400
14114993523,8cyclictest252812chrt00:00:273
14114993523,8cyclictest252812chrt00:00:273
14114993523,8cyclictest252812chrt00:00:273
1411099351,32cyclictest9917-21cut20:25:311
1411099351,32cyclictest1799-21ntp_states22:55:391
1411099351,32cyclictest1799-21ntp_states22:55:391
1411099351,32cyclictest0-21swapper/123:29:341
1411099351,32cyclictest0-21swapper/123:29:341
1411099351,31cyclictest25024-21mailstats22:30:391
1411099351,31cyclictest22421-21fschecks_count22:25:311
1411099351,31cyclictest18477-21users20:45:441
1411099351,31cyclictest14050-21memory20:35:341
1411099351,31cyclictest14050-21memory20:35:341
1411099351,31cyclictest10578-21sshd21:52:501
1411099351,31cyclictest10578-21sshd21:52:501
1411099351,31cyclictest0-21swapper/100:40:061
1411099351,31cyclictest0-21swapper/100:40:061
1411099351,30cyclictest26983-21latency_hist19:45:221
1411099351,30cyclictest26983-21latency_hist19:45:221
1411099350,33cyclictest6248-21sshd23:07:501
1411099350,33cyclictest6248-21sshd23:07:501
1411099350,33cyclictest32231-21/usr/sbin/munin00:15:431
1411099350,33cyclictest32231-21/usr/sbin/munin00:15:431
1411099350,33cyclictest32231-21/usr/sbin/munin00:15:431
1411099350,33cyclictest31750-21sshd21:22:491
1411099350,33cyclictest31750-21sshd21:22:491
1411099350,32cyclictest29361-21if_err_eth019:50:331
1411099350,25cyclictest19599-21wc20:50:351
1411099350,25cyclictest19599-21wc20:50:351
315732340,3chrt0-21swapper/319:55:393
315732340,3chrt0-21swapper/319:55:393
315732340,3chrt0-21swapper/319:55:393
1411099341,31cyclictest7397-21/usr/sbin/munin21:45:321
1411099341,31cyclictest7397-21/usr/sbin/munin21:45:321
1411099341,31cyclictest4216-21/usr/sbin/munin00:30:411
1411099341,31cyclictest4216-21/usr/sbin/munin00:30:411
1411099341,31cyclictest27852-21ntp_states19:45:381
1411099341,31cyclictest11748-21ntp_states21:55:391
1411099341,31cyclictest11748-21ntp_states21:55:391
1411099341,31cyclictest0-21swapper/123:57:351
1411099341,30cyclictest3097-21ntp_states00:25:381
1411099341,30cyclictest3097-21ntp_states00:25:381
14107993426,6cyclictest5859-21latency_hist00:35:180
14107993426,6cyclictest5859-21latency_hist00:35:180
1411099331,30cyclictest20250-21irqstats23:45:341
1411099331,30cyclictest20250-21irqstats23:45:341
1411099331,29cyclictest6902-21taskset21:43:271
1411099331,29cyclictest6902-21taskset21:43:271
1411099331,29cyclictest6102-21/usr/sbin/munin20:15:321
1411099331,29cyclictest6102-21/usr/sbin/munin20:15:321
1411099331,29cyclictest19069-21missed_timers22:15:351
1411099331,29cyclictest19069-21missed_timers22:15:351
1411099331,29cyclictest11318-21cut23:20:391
1411099331,29cyclictest11318-21cut23:20:391
14110993310,21cyclictest19662-21cpu19:25:291
14110993310,21cyclictest19662-21cpu19:25:291
1411099330,29cyclictest16688-21sort22:10:321
1411099330,29cyclictest16688-21sort22:10:321
1411099330,23cyclictest12881-21/usr/sbin/munin22:00:341
1411099330,23cyclictest12881-21/usr/sbin/munin22:00:341
26522320,1sleep02653-21sshd22:59:500
26522320,1sleep02653-21sshd22:59:500
1411099321,29cyclictest0-21swapper/122:38:291
1411099321,29cyclictest0-21swapper/122:38:291
1411099321,29cyclictest0-21swapper/122:38:291
1411099321,29cyclictest0-21swapper/119:36:571
1411099321,29cyclictest0-21swapper/119:36:571
1411099321,28cyclictest4063-21sed23:00:411
1411099321,28cyclictest3866-21gltestperf21:35:321
1411099321,28cyclictest3866-21gltestperf21:35:321
1411099321,28cyclictest3866-21gltestperf21:35:321
1411099321,26cyclictest19458-21taskset23:44:181
1411099321,26cyclictest19458-21taskset23:44:181
14107993224,6cyclictest9197-21needreboot23:15:360
14107993224,6cyclictest9197-21needreboot23:15:360
14107993222,8cyclictest9835-21netstat21:50:360
14107993222,8cyclictest9835-21netstat21:50:360
297092310,3sleep00-21swapper/019:50:380
184202310,2sleep018422-21grep23:40:340
184202310,2sleep018422-21grep23:40:340
14107993124,5cyclictest638-21sshd22:53:500
14107993124,5cyclictest638-21sshd22:53:500
14107993120,8cyclictest24581-21kernelversion22:30:330
229362300,2sleep022937-21/usr/sbin/munin22:25:360
158312300,2sleep015834-21wc22:05:420
158312300,2sleep015834-21wc22:05:420
14107993024,5cyclictest28113-21threads19:45:410
14107993024,5cyclictest28113-21threads19:45:410
14107993022,6cyclictest6576-21kernelversion00:35:340
14107993022,6cyclictest6576-21kernelversion00:35:340
14107993022,6cyclictest19976-21grep20:50:410
14107993022,6cyclictest19976-21grep20:50:410
14107992924,4cyclictest2027-21latency_hist20:05:190
14107992924,4cyclictest2027-21latency_hist20:05:190
14107992923,3cyclictest0-21swapper/019:30:180
14107992923,3cyclictest0-21swapper/019:30:180
14107992921,6cyclictest8543-21needreboot20:20:360
14107992920,7cyclictest24813-21fschecks_count21:05:310
14107992918,8cyclictest32003-21latency_hist21:25:200
14107992918,8cyclictest32003-21latency_hist21:25:200
323092280,3sleep20-21swapper/222:50:382
323092280,3sleep20-21swapper/222:50:382
14572270,4chrt0-21swapper/322:55:343
14572270,4chrt0-21swapper/322:55:343
14107992721,4cyclictest25474-21fschecks_time19:40:340
14107992721,4cyclictest25474-21fschecks_time19:40:340
14107992721,3cyclictest0-21swapper/021:45:350
14107992721,3cyclictest0-21swapper/021:45:350
14107992720,5cyclictest5318-21unixbench_multi20:10:410
14107992720,5cyclictest5318-21unixbench_multi20:10:410
14107992720,5cyclictest5318-21unixbench_multi20:10:410
14107992720,5cyclictest3568-21sshd20:06:490
14107992720,5cyclictest3568-21sshd20:06:490
14107992720,5cyclictest31273-21gltestperf00:15:340
14107992720,5cyclictest31273-21gltestperf00:15:340
14107992720,5cyclictest31273-21gltestperf00:15:340
14107992720,5cyclictest21905-21cut23:50:320
14107992720,5cyclictest18443-21needreboot19:20:370
14107992720,5cyclictest18443-21needreboot19:20:370
14107992720,5cyclictest10585-21rt-features20:25:390
14107992718,7cyclictest29494-21latency_hist22:45:180
14107992718,7cyclictest29494-21latency_hist22:45:180
14107992717,8cyclictest26706-21fschecks_count21:10:310
14107992714,10cyclictest24454-21cron21:05:180
14107992711,13cyclictest2354-21/usr/sbin/munin00:25:350
14107992711,13cyclictest2354-21/usr/sbin/munin00:25:350
14107992621,3cyclictest0-21swapper/021:40:400
14107992621,3cyclictest0-21swapper/021:40:400
14107992621,2cyclictest0-21swapper/020:40:400
14107992621,2cyclictest0-21swapper/020:40:400
14107992619,5cyclictest13926-21latency_hist23:30:180
14107992619,5cyclictest13926-21latency_hist23:30:180
14107992619,4cyclictest0-21swapper/023:20:400
14107992619,4cyclictest0-21swapper/023:20:400
14107992619,4cyclictest0-21swapper/022:35:380
14107992619,4cyclictest0-21swapper/022:35:380
14107992619,4cyclictest0-21swapper/022:35:380
14107992617,6cyclictest15271-21latency_hist20:40:250
14107992617,6cyclictest15271-21latency_hist20:40:250
14107992614,10cyclictest142-21systemd-journal22:15:180
14107992614,10cyclictest142-21systemd-journal22:15:180
14107992613,11cyclictest4576-21cron23:05:180
14107992519,4cyclictest749-21runrttasks23:30:380
14107992519,4cyclictest749-21runrttasks23:30:380
14107992519,3cyclictest0-21swapper/022:15:330
14107992519,3cyclictest0-21swapper/022:15:330
14107992517,5cyclictest0-21swapper/021:00:180
14107992517,5cyclictest0-21swapper/021:00:180
14107992515,8cyclictest28910-21kernelversion21:15:330
14107992515,8cyclictest28910-21kernelversion21:15:330
14107992515,8cyclictest28910-21kernelversion21:15:330
14107992511,11cyclictest10994-21date21:55:280
14107992511,11cyclictest10994-21date21:55:280
14107992510,12cyclictest0-21swapper/021:25:310
14107992510,12cyclictest0-21swapper/021:25:310
14107992418,5cyclictest20881-21timerwakeupswit23:45:420
14107992418,5cyclictest20881-21timerwakeupswit23:45:420
14107992418,3cyclictest0-21swapper/020:30:320
14107992417,4cyclictest0-21swapper/022:20:400
14107992417,4cyclictest0-21swapper/022:20:400
14107992416,6cyclictest29437-21head00:10:320
14107992416,6cyclictest29437-21head00:10:320
14107992317,4cyclictest13797-21systemctl22:00:410
14107992317,4cyclictest13797-21systemctl22:00:410
14107992316,5cyclictest14102-21cyclictest23:35:320
14107992316,5cyclictest14102-21cyclictest23:35:320
14107992315,6cyclictest6631-21ls23:10:260
14107992315,6cyclictest6631-21ls23:10:260
14107992313,7cyclictest3526-21/usr/sbin/munin21:35:300
14107992313,7cyclictest3526-21/usr/sbin/munin21:35:300
14107992313,7cyclictest3526-21/usr/sbin/munin21:35:300
14107992215,5cyclictest1834-21fschecks_count21:30:320
14107992215,5cyclictest1834-21fschecks_count21:30:320
14107992115,3cyclictest0-21swapper/020:15:430
14107992115,3cyclictest0-21swapper/020:15:430
14107992115,3cyclictest0-21swapper/019:35:330
14107992115,3cyclictest0-21swapper/019:35:330
1410799211,4cyclictest201rcuc/020:45:360
269502200,2sleep20-21swapper/222:35:392
269502200,2sleep20-21swapper/222:35:392
269502200,2sleep20-21swapper/222:35:392
1411399201,12cyclictest24080-21chrt19:35:362
1411399201,12cyclictest24080-21chrt19:35:362
14107992015,3cyclictest0-21swapper/022:45:410
14107992015,3cyclictest0-21swapper/022:45:410
1411499193,9cyclictest0-21swapper/319:31:483
1411499193,9cyclictest0-21swapper/300:36:503
1411499193,9cyclictest0-21swapper/300:36:503
14114991911,6cyclictest28919-21chrt22:40:423
14114991911,6cyclictest28919-21chrt22:40:423
1411499191,13cyclictest20815-21chrt22:20:323
1411499191,13cyclictest20815-21chrt22:20:323
1411399193,10cyclictest0-21swapper/200:00:352
1411399193,10cyclictest0-21swapper/200:00:352
1411499181,12cyclictest0-21swapper/323:15:303
1411499181,12cyclictest0-21swapper/323:15:303
1411499181,11cyclictest668-21chrt00:20:333
1411499181,11cyclictest668-21chrt00:20:333
1411399181,13cyclictest32342-21chrt21:25:322
1411399181,13cyclictest32342-21chrt21:25:322
1411399181,13cyclictest14452-21taskset23:30:302
1411399181,13cyclictest14452-21taskset23:30:302
1411399181,13cyclictest1334-21chrt22:55:332
1411399181,13cyclictest1334-21chrt22:55:332
1411499179,5cyclictest294312sleep321:15:413
1411499179,5cyclictest294312sleep321:15:413
1411499179,5cyclictest294312sleep321:15:413
1411499173,9cyclictest0-21swapper/323:52:403
14114991712,3cyclictest167742chrt23:35:383
14114991712,3cyclictest167742chrt23:35:383
1411499171,12cyclictest681-21chrt21:25:413
1411499171,12cyclictest681-21chrt21:25:413
1411499171,12cyclictest15213-21chrt23:30:403
1411499171,12cyclictest15213-21chrt23:30:403
1411499171,11cyclictest2457-21taskset00:25:313
1411499171,11cyclictest2457-21taskset00:25:313
1411499171,11cyclictest15688-21chrt22:05:413
1411499171,11cyclictest15688-21chrt22:05:413
1411399172,9cyclictest0-21swapper/200:25:502
1411399172,9cyclictest0-21swapper/200:25:502
1411399171,12cyclictest8110-21taskset23:12:502
1411399171,12cyclictest8110-21taskset23:12:502
1411399171,12cyclictest22334-21chrt23:50:372
1411399171,11cyclictest8772-21chrt20:20:392
1411399171,11cyclictest30300-21taskset21:20:292
1411399171,11cyclictest30300-21taskset21:20:292
1411399171,11cyclictest20656-21taskset22:20:312
1411399171,11cyclictest20656-21taskset22:20:312
1411399171,11cyclictest18088-21chrt19:20:332
1411399171,11cyclictest18088-21chrt19:20:332
1411399171,11cyclictest1544-21taskset20:00:412
1411399171,11cyclictest1544-21taskset20:00:412
1411499163,9cyclictest0-21swapper/319:35:493
1411499163,9cyclictest0-21swapper/319:35:493
1411499161,9cyclictest4805-21taskset00:30:383
1411499161,9cyclictest4805-21taskset00:30:383
1411499161,11cyclictest19825-21chrt20:50:393
1411499161,11cyclictest19825-21chrt20:50:393
1411499161,11cyclictest0-21swapper/323:30:263
1411499161,11cyclictest0-21swapper/323:30:263
1411499161,11cyclictest0-21swapper/322:50:353
1411499161,11cyclictest0-21swapper/322:50:353
1411499161,11cyclictest0-21swapper/321:55:333
1411499161,11cyclictest0-21swapper/321:55:333
1411499161,11cyclictest0-21swapper/320:35:313
1411499161,11cyclictest0-21swapper/320:35:313
1411499161,11cyclictest0-21swapper/320:30:423
1411499161,10cyclictest9883-21chrt21:50:373
1411499161,10cyclictest9883-21chrt21:50:373
1411499161,10cyclictest3548-21taskset21:35:303
1411499161,10cyclictest3548-21taskset21:35:303
1411499161,10cyclictest3548-21taskset21:35:303
1411499161,10cyclictest2431-21chrt21:30:393
1411499161,10cyclictest2431-21chrt21:30:393
1411499161,10cyclictest22397-21taskset22:25:313
1411499161,10cyclictest0-21swapper/320:00:403
1411499161,10cyclictest0-21swapper/320:00:403
1411399163,9cyclictest0-21swapper/220:51:492
1411399163,9cyclictest0-21swapper/220:51:492
1411399161,11cyclictest21112-21chrt20:55:322
1411399161,11cyclictest21112-21chrt20:55:322
1411399161,11cyclictest0-21swapper/221:55:302
1411399161,11cyclictest0-21swapper/221:55:302
1411399161,10cyclictest13574-21chrt20:35:302
1411399161,10cyclictest13574-21chrt20:35:302
1411399161,10cyclictest10975-21chrt21:55:272
1411399161,10cyclictest10975-21chrt21:55:272
1411499151,9cyclictest0-21swapper/300:15:333
1411499151,9cyclictest0-21swapper/300:15:333
1411499151,9cyclictest0-21swapper/300:15:333
1411499151,8cyclictest0-21swapper/320:20:303
1411499151,11cyclictest13625-21chrt22:00:383
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional