You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-05-09 - 07:20

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #4, slot #1

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1s.osadl.org (updated Fri May 09, 2025 00:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
73792800,3sleep20-21swapper/221:40:472
228922720,3sleep30-21swapper/320:55:503
228922720,3sleep30-21swapper/320:55:503
104762640,3sleep30-21swapper/321:50:293
104762640,3sleep30-21swapper/321:50:293
14940996320,39cyclictest0-21swapper/023:05:420
14940996320,39cyclictest0-21swapper/023:05:420
122882620,2sleep00-21swapper/023:20:470
246472590,3sleep20-21swapper/221:00:472
246472590,3sleep20-21swapper/221:00:472
1494099599,13cyclictest181rcu_preempt20:00:400
1494099599,13cyclictest181rcu_preempt20:00:400
29132580,3sleep30-21swapper/322:55:453
256982580,1sleep125875-21head23:55:461
256982580,1sleep125875-21head23:55:461
123762580,2chrt0-21swapper/123:20:471
36522570,2sleep13772-21timerandwakeup22:55:561
1494099564,9cyclictest181rcu_preempt22:00:440
1494099564,9cyclictest181rcu_preempt22:00:440
1494099564,9cyclictest181rcu_preempt21:00:430
1494099564,9cyclictest181rcu_preempt21:00:430
47192540,3sleep10-21swapper/120:05:551
205112540,2sleep120592-21fschecks_count20:50:421
14941995419,12cyclictest27965-21apt-get21:10:431
14941995419,12cyclictest27965-21apt-get21:10:431
105352530,3sleep30-21swapper/323:15:433
296592520,2sleep30-21swapper/321:15:313
296592520,2sleep30-21swapper/321:15:313
268022520,4sleep30-21swapper/321:05:513
268022520,4sleep30-21swapper/321:05:513
14941995219,13cyclictest29543-21apt-get22:40:431
14941995219,13cyclictest29543-21apt-get22:40:431
1494099513,11cyclictest181rcu_preempt00:00:450
1494099513,11cyclictest181rcu_preempt00:00:450
1494099502,36cyclictest6420-21taskset00:30:420
1494099502,36cyclictest6420-21taskset00:30:420
1494099493,8cyclictest181rcu_preempt21:10:430
1494099493,8cyclictest181rcu_preempt21:10:430
266372480,3sleep30-21swapper/322:30:533
212582480,3sleep20-21swapper/223:41:472
14941994838,5cyclictest0-21swapper/123:30:381
14941994814,23cyclictest0-21swapper/120:20:391
14941994814,23cyclictest0-21swapper/120:20:391
129032480,2sleep112909-21cpu20:30:431
129032480,2sleep112909-21cpu20:30:431
117392480,3sleep30-21swapper/321:50:573
1494099479,27cyclictest55150irq/47-eth022:35:430
1494099479,27cyclictest55150irq/47-eth022:35:430
1494099464,11cyclictest181rcu_preempt22:40:430
1494099464,11cyclictest181rcu_preempt22:40:430
14940994623,17cyclictest171ktimers/000:05:420
14940994623,17cyclictest171ktimers/000:05:420
71992450,2sleep00-21swapper/020:15:400
71992450,2sleep00-21swapper/020:15:400
1494099451,9cyclictest181rcu_preempt21:40:420
301872440,5sleep030190-21cut21:15:440
1494199444,36cyclictest663-21runrttasks20:40:411
1494199444,36cyclictest663-21runrttasks20:40:411
1494099441,9cyclictest181rcu_preempt19:35:420
1494099441,9cyclictest181rcu_preempt19:35:420
292962430,3sleep114939-21cyclictest19:45:531
179842430,2sleep117985-21cut23:35:391
161542430,6chrt201rcuc/023:30:430
161542430,6chrt201rcuc/023:30:430
14941994310,27cyclictest0-21swapper/123:45:421
14941994310,27cyclictest0-21swapper/123:45:421
1494099431,36cyclictest394-21snmpd21:20:420
1494099431,36cyclictest394-21snmpd21:20:420
14941994218,17cyclictest281ktimers/120:15:401
14941994218,17cyclictest281ktimers/120:15:401
1494099426,27cyclictest55150irq/47-eth019:55:400
1494099426,27cyclictest55150irq/47-eth019:55:400
14940994228,10cyclictest0-21swapper/019:25:420
14940994228,10cyclictest0-21swapper/019:25:420
1494099422,4cyclictest181rcu_preempt00:20:410
1494099422,4cyclictest181rcu_preempt00:20:410
14941994112,21cyclictest0-21swapper/123:40:421
14940994133,6cyclictest17053-21netstat22:05:510
14940994133,6cyclictest17053-21netstat22:05:510
1494099402,3cyclictest181rcu_preempt20:45:430
1494099402,3cyclictest181rcu_preempt20:45:430
148172400,3sleep30-21swapper/323:25:523
142862400,1sleep10-21swapper/122:00:321
142862400,1sleep10-21swapper/122:00:321
108102400,2sleep110973-21perl21:50:481
14941993930,5cyclictest20111-21apt-get22:15:431
14941993930,5cyclictest20111-21apt-get22:15:431
14941993823,10cyclictest29438-21apt-get00:05:421
14941993823,10cyclictest29438-21apt-get00:05:421
1494099387,22cyclictest55150irq/47-eth021:35:410
1494099387,22cyclictest55150irq/47-eth021:35:410
14940993825,10cyclictest0-21swapper/022:20:400
14940993825,10cyclictest0-21swapper/022:20:400
284732370,2sleep028570-21fschecks_time19:45:450
14941993724,10cyclictest2662-21apt-get00:20:411
14941993724,10cyclictest2662-21apt-get00:20:411
14941993712,11cyclictest9533-21ntp_kernel_pll_21:45:501
14941993712,11cyclictest9533-21ntp_kernel_pll_21:45:501
14941993712,10cyclictest23017-21threads22:20:521
14941993712,10cyclictest23017-21threads22:20:521
14941993710,22cyclictest0-21swapper/120:35:381
1494099376,24cyclictest55150irq/47-eth020:25:420
1494099376,24cyclictest55150irq/47-eth020:25:420
1494099375,26cyclictest55150irq/47-eth022:15:420
1494099375,26cyclictest55150irq/47-eth022:15:420
1494099374,12cyclictest181rcu_preempt22:55:410
1494199366,11cyclictest31660-21fschecks_time22:45:451
1494199366,11cyclictest31660-21fschecks_time22:45:451
14941993627,5cyclictest16150-21apt-get23:30:441
14941993624,10cyclictest10367-21apt-get23:15:411
14941993611,14cyclictest18976-21ntp_states22:10:511
14941993611,14cyclictest18976-21ntp_states22:10:511
325052350,3sleep20-21swapper/221:20:532
325052350,3sleep20-21swapper/221:20:532
1494299352,25cyclictest0-21swapper/223:05:422
1494299352,25cyclictest0-21swapper/223:05:422
14941993521,11cyclictest24195-21apt-get21:00:421
14941993521,11cyclictest24195-21apt-get21:00:421
14941993514,9cyclictest746-21apt-get00:15:441
14941993514,9cyclictest746-21apt-get00:15:441
14940993524,7cyclictest0-21swapper/021:50:440
236302340,2sleep123652-21latency_hist23:50:391
236302340,2sleep123652-21latency_hist23:50:391
14941993424,8cyclictest29837-21apt-get21:15:421
14941993419,10cyclictest18543-21apt-get20:45:431
14941993419,10cyclictest18543-21apt-get20:45:431
1494099349,21cyclictest171ktimers/020:30:410
1494099349,21cyclictest171ktimers/020:30:410
1494099346,10cyclictest181rcu_preempt21:25:460
1494099346,10cyclictest181rcu_preempt21:25:460
14940993427,5cyclictest60-21kcompactd023:30:380
282112330,2sleep1281ktimers/119:45:331
258762330,1sleep20-21swapper/223:55:462
258762330,1sleep20-21swapper/223:55:462
1494299332,23cyclictest0-21swapper/220:05:422
1494299331,25cyclictest0-21swapper/200:15:432
1494299331,25cyclictest0-21swapper/200:15:432
14941993311,9cyclictest24489-21apt-get19:35:421
14941993311,9cyclictest24489-21apt-get19:35:421
1494199328,11cyclictest23176-21cat19:30:481
1494199328,11cyclictest23176-21cat19:30:481
1494199325,21cyclictest25533-21kworker/1:2+events21:05:411
1494199325,21cyclictest25533-21kworker/1:2+events21:05:411
14941993225,4cyclictest27564-21apt-get00:00:451
14941993225,4cyclictest27564-21apt-get00:00:451
14941993220,9cyclictest14475-21apt-get22:00:431
14941993220,9cyclictest14475-21apt-get22:00:431
14941993212,9cyclictest2384-21wc20:00:501
14941993212,9cyclictest2384-21wc20:00:501
14941993210,5cyclictest11006-21apt-get20:25:431
14941993210,5cyclictest11006-21apt-get20:25:431
14940993225,5cyclictest18456-21cut22:10:440
14940993225,5cyclictest18456-21cut22:10:440
58122310,3sleep15818-21timerwakeupswit23:00:541
58122310,3sleep15818-21timerwakeupswit23:00:541
4292310,1sleep1536-21/usr/sbin/munin19:55:511
4292310,1sleep1536-21/usr/sbin/munin19:55:511
1494299311,23cyclictest0-21swapper/200:25:412
1494299311,23cyclictest0-21swapper/200:25:412
1494299311,22cyclictest0-21swapper/222:45:432
1494299311,22cyclictest0-21swapper/222:45:432
1494199318,10cyclictest27626-21taskset22:35:411
1494199318,10cyclictest27626-21taskset22:35:411
1494199316,11cyclictest19470-21seq23:38:111
1494199316,11cyclictest19470-21seq23:38:111
14941993123,7cyclictest30406-21fschecks_count19:50:421
14941993123,7cyclictest30406-21fschecks_count19:50:421
14941993122,7cyclictest0-21swapper/122:06:061
14941993122,7cyclictest0-21swapper/122:06:061
14941993122,6cyclictest20627-21apt-get19:25:421
14941993122,6cyclictest20627-21apt-get19:25:421
14941993120,9cyclictest4940-21apt-get21:35:431
14941993120,9cyclictest4940-21apt-get21:35:431
14941993120,8cyclictest0-21swapper/122:53:471
14941993120,8cyclictest0-21swapper/122:53:471
14941993119,8cyclictest0-21swapper/120:55:431
14941993119,8cyclictest0-21swapper/120:55:431
14941993119,8cyclictest0-21swapper/100:35:411
14941993119,8cyclictest0-21swapper/100:35:411
1494099317,19cyclictest55150irq/47-eth000:10:430
1494099317,19cyclictest55150irq/47-eth000:10:430
1494299303,20cyclictest0-21swapper/221:50:442
1494299303,18cyclictest0-21swapper/222:40:432
1494299303,18cyclictest0-21swapper/222:40:432
1494299302,23cyclictest0-21swapper/220:25:422
1494299302,23cyclictest0-21swapper/220:25:422
1494299302,22cyclictest0-21swapper/222:00:432
1494299302,22cyclictest0-21swapper/222:00:432
1494299302,21cyclictest0-21swapper/219:55:402
1494299302,21cyclictest0-21swapper/219:55:402
1494299301,23cyclictest0-21swapper/219:40:402
1494299300,23cyclictest31747-21chrt19:53:502
1494299300,23cyclictest31747-21chrt19:53:502
14941993024,4cyclictest31750-21apt-get21:20:421
14941993024,4cyclictest31750-21apt-get21:20:421
14941993023,5cyclictest6966-21head23:05:461
14941993023,5cyclictest6966-21head23:05:461
14941993023,5cyclictest5541-21threads00:25:541
14941993023,5cyclictest5541-21threads00:25:541
1494099304,9cyclictest181rcu_preempt23:55:410
1494099304,9cyclictest181rcu_preempt23:55:410
14940993024,5cyclictest9000-21ntp_kernel_pll_00:35:500
14940993024,5cyclictest9000-21ntp_kernel_pll_00:35:500
14940993023,5cyclictest32034-21missed_timers22:45:480
14940993023,5cyclictest32034-21missed_timers22:45:480
1494299293,19cyclictest0-21swapper/222:20:402
1494299293,19cyclictest0-21swapper/222:20:402
1494299292,21cyclictest0-21swapper/223:45:422
1494299292,21cyclictest0-21swapper/223:45:422
1494299292,20cyclictest0-21swapper/220:55:432
1494299292,20cyclictest0-21swapper/220:55:432
1494299291,24cyclictest32724-21chrt00:12:462
1494299291,24cyclictest32724-21chrt00:12:462
1494299291,24cyclictest15815-21chrt23:30:292
1494299291,23cyclictest6142-21chrt23:02:352
1494299291,23cyclictest6142-21chrt23:02:352
1494299291,23cyclictest14271-21chrt20:32:292
1494299291,23cyclictest14271-21chrt20:32:292
1494299291,22cyclictest0-21swapper/221:45:422
1494299291,22cyclictest0-21swapper/221:45:422
1494299291,21cyclictest0-21swapper/223:10:412
1494299291,21cyclictest0-21swapper/223:10:412
1494299291,20cyclictest0-21swapper/223:30:452
1494299291,20cyclictest0-21swapper/223:30:452
14941992923,5cyclictest26949-21unixbench_multi22:30:551
14941992917,9cyclictest5823-21irqstats20:10:471
14941992917,9cyclictest5823-21irqstats20:10:471
14940992922,5cyclictest27503-21timerandwakeup19:40:520
14940992922,5cyclictest19659-21latency_hist23:40:290
14940992922,5cyclictest19659-21latency_hist23:40:290
14940992920,7cyclictest25929-21fschecks_count22:30:430
1494299283,18cyclictest0-21swapper/221:30:412
1494299283,18cyclictest0-21swapper/221:30:412
1494299282,20cyclictest0-21swapper/220:40:402
1494299282,20cyclictest0-21swapper/220:40:402
1494299282,19cyclictest0-21swapper/222:50:412
1494299282,19cyclictest0-21swapper/222:50:412
1494299282,18cyclictest0-21swapper/222:15:422
1494299282,18cyclictest0-21swapper/222:15:422
1494299281,23cyclictest4173-21chrt22:58:052
1494299281,23cyclictest27573-21chrt21:08:552
1494299281,23cyclictest27573-21chrt21:08:552
1494299281,23cyclictest25394-21chrt22:28:512
1494299281,23cyclictest25394-21chrt22:28:512
1494299281,23cyclictest22233-21chrt19:30:142
1494299281,23cyclictest22233-21chrt19:30:142
1494299281,23cyclictest21728-21chrt20:51:302
1494299281,23cyclictest0-21swapper/223:15:412
1494299281,22cyclictest2552-21taskset21:27:082
1494299281,22cyclictest2552-21taskset21:27:082
1494299281,21cyclictest239562sleep219:31:562
1494299281,21cyclictest239562sleep219:31:562
1494299281,18cyclictest0-21swapper/200:00:452
1494299281,18cyclictest0-21swapper/200:00:452
1494299280,24cyclictest10499-21chrt20:22:162
1494299280,24cyclictest10499-21chrt20:22:162
1494299280,23cyclictest27648-21chrt22:35:422
1494299280,23cyclictest27648-21chrt22:35:422
1494299280,23cyclictest17848-21chrt22:08:452
1494299280,23cyclictest17848-21chrt22:08:452
1494299280,23cyclictest16329-21chrt20:39:412
1494199283,21cyclictest394-21snmpd21:30:421
1494199283,21cyclictest394-21snmpd21:30:421
14941992820,6cyclictest0-21swapper/121:42:461
14941992817,9cyclictest2382-21awk21:26:011
14941992817,9cyclictest2382-21awk21:26:011
14941992816,10cyclictest6417-21apt-get00:30:431
14941992816,10cyclictest6417-21apt-get00:30:431
14941992815,10cyclictest31314-21apt-get00:10:421
14941992815,10cyclictest31314-21apt-get00:10:421
14940992822,5cyclictest5022-21kernelversion00:25:470
14940992822,5cyclictest5022-21kernelversion00:25:470
14940992820,6cyclictest24020-21fschecks_count22:25:450
14940992820,6cyclictest24020-21fschecks_count22:25:450
14940992815,11cyclictest0-21swapper/023:40:450
1494399272,19cyclictest27649-21taskset22:35:433
1494399272,19cyclictest27649-21taskset22:35:433
1494299272,21cyclictest0-21swapper/219:45:422
1494299272,20cyclictest0-21swapper/200:30:412
1494299272,20cyclictest0-21swapper/200:30:412
1494299272,18cyclictest0-21swapper/220:45:432
1494299272,18cyclictest0-21swapper/220:45:432
1494299271,22cyclictest8548-21chrt20:16:412
1494299271,22cyclictest8548-21chrt20:16:412
1494299271,22cyclictest30725-21chrt00:06:112
1494299271,22cyclictest30725-21chrt00:06:112
1494299271,22cyclictest29274-21chrt21:11:282
1494299271,22cyclictest29274-21chrt21:11:282
1494299271,22cyclictest27054-21chrt22:31:082
1494299271,22cyclictest26041-21chrt19:39:352
1494299271,22cyclictest26041-21chrt19:39:352
1494299271,20cyclictest0-21swapper/200:35:412
1494299271,20cyclictest0-21swapper/200:35:412
1494299270,23cyclictest14166-21chrt22:00:002
1494299270,23cyclictest14166-21chrt22:00:002
1494299270,22cyclictest3964-21chrt00:21:222
1494299270,22cyclictest3964-21chrt00:21:222
1494299270,22cyclictest19299-21chrt23:36:002
1494299270,22cyclictest19299-21chrt23:36:002
1494299270,22cyclictest13589-21chrt23:21:292
14941992715,9cyclictest0-21swapper/123:10:411
14941992715,9cyclictest0-21swapper/123:10:411
14940992721,4cyclictest663-21runrttasks23:00:410
14940992721,4cyclictest663-21runrttasks23:00:410
14940992720,5cyclictest25010-21sshd23:51:470
14940992720,5cyclictest25010-21sshd23:51:470
14940992719,6cyclictest22808-21fschecks_time19:30:450
14940992719,6cyclictest22808-21fschecks_time19:30:450
242752260,3sleep30-21swapper/322:25:463
242752260,3sleep30-21swapper/322:25:463
1494299263,17cyclictest0-21swapper/220:00:402
1494299263,17cyclictest0-21swapper/220:00:402
1494299262,18cyclictest0-21swapper/221:35:412
1494299262,18cyclictest0-21swapper/221:35:412
1494299261,3cyclictest0-21swapper/221:15:422
1494299261,21cyclictest6651-21chrt20:11:272
1494299261,21cyclictest6651-21chrt20:11:272
1494299261,21cyclictest19687-21chrt22:13:152
1494299261,21cyclictest19687-21chrt22:13:152
1494299261,20cyclictest0-21swapper/223:50:402
1494299261,20cyclictest0-21swapper/223:50:402
14940992620,5cyclictest5028-21sshd20:09:400
14940992619,5cyclictest20688-21gltestperf20:50:430
14941992514,9cyclictest0-21swapper/122:25:411
14941992514,9cyclictest0-21swapper/122:25:411
14940992521,2cyclictest0-21swapper/000:15:480
14940992521,2cyclictest0-21swapper/000:15:480
14940992520,3cyclictest0-21swapper/023:10:450
14940992520,3cyclictest0-21swapper/023:10:450
14940992519,4cyclictest14939-21cyclictest23:46:320
14940992519,4cyclictest14939-21cyclictest23:46:320
14940992519,4cyclictest14939-21cyclictest20:14:310
14940992519,4cyclictest14939-21cyclictest20:14:310
14940992518,5cyclictest26440-21idleruntime21:05:460
14940992518,5cyclictest26440-21idleruntime21:05:460
14940992419,3cyclictest0-21swapper/023:20:310
14940992419,3cyclictest0-21swapper/020:45:290
14940992419,3cyclictest0-21swapper/020:45:290
14940992414,8cyclictest9920-21processes20:20:490
14940992414,8cyclictest9920-21processes20:20:490
1494399233,11cyclictest0-21swapper/323:00:423
1494399233,11cyclictest0-21swapper/323:00:423
1494099232,16cyclictest8829-21seq21:45:430
1494099232,16cyclictest8829-21seq21:45:430
14940992319,3cyclictest15115-21head20:35:430
14940992317,3cyclictest0-21swapper/022:50:500
14940992317,3cyclictest0-21swapper/022:50:500
1494399222,10cyclictest72252sleep320:15:423
1494399222,10cyclictest72252sleep320:15:423
1494099227,12cyclictest16-21ksoftirqd/019:50:400
1494099227,12cyclictest16-21ksoftirqd/019:50:400
1494099225,2cyclictest181rcu_preempt20:55:430
1494099225,2cyclictest181rcu_preempt20:55:430
14940992216,4cyclictest12800-21cut21:55:420
14940992216,4cyclictest12800-21cut21:55:420
14940992216,3cyclictest0-21swapper/021:30:500
14940992216,3cyclictest0-21swapper/021:30:500
1494399203,9cyclictest0-21swapper/319:40:383
1494399192,9cyclictest0-21swapper/322:40:433
1494399192,9cyclictest0-21swapper/322:40:433
1494399183,9cyclictest0-21swapper/321:45:363
29432170,3sleep30-21swapper/321:30:353
29432170,3sleep30-21swapper/321:30:353
1494399173,8cyclictest0-21swapper/322:10:423
1494399173,8cyclictest0-21swapper/322:10:423
1494399173,10cyclictest0-21swapper/320:51:403
1494399172,9cyclictest0-21swapper/300:00:353
1494399172,9cyclictest0-21swapper/300:00:353
1494399171,13cyclictest20862-21taskset19:25:453
1494399171,13cyclictest20862-21taskset19:25:453
1494399171,12cyclictest28378-21taskset19:45:433
1494399171,10cyclictest0-21swapper/320:00:413
1494399171,10cyclictest0-21swapper/320:00:413
1494399166,5cyclictest0-21swapper/321:15:413
1494399163,9cyclictest0-21swapper/323:35:433
1494399163,9cyclictest0-21swapper/323:35:433
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional