You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-21 - 07:27
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1.osadl.org (updated Fri Nov 21, 2025 00:44:59)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1616321300,3sleep30-21swapper/300:00:473
1616321300,3sleep30-21swapper/300:00:473
2955921120,8sleep333-21ksoftirqd/323:02:073
2955921120,8sleep333-21ksoftirqd/323:02:073
1088121090,9sleep333-21ksoftirqd/323:54:563
1088121090,9sleep333-21ksoftirqd/323:54:563
1397321060,3sleep30-21swapper/321:29:443
2233021000,2sleep30-21swapper/319:21:013
2233021000,2sleep30-21swapper/319:21:013
46302940,3sleep30-21swapper/321:56:043
46302940,3sleep30-21swapper/321:56:043
88752900,4chrt0-21swapper/322:39:493
88752900,4chrt0-21swapper/322:39:493
65472850,4sleep10-21swapper/100:27:091
65472850,4sleep10-21swapper/100:27:091
286212840,2sleep10-21swapper/100:15:401
286212840,2sleep10-21swapper/100:15:401
312442810,3sleep20-21swapper/223:04:522
312442810,3sleep20-21swapper/223:04:522
79602740,4chrt0-21swapper/222:00:502
79602740,4chrt0-21swapper/222:00:502
315732710,3chrt0-21swapper/223:41:052
315732710,3chrt0-21swapper/223:41:052
250492700,3sleep20-21swapper/221:41:332
250492700,3sleep20-21swapper/221:41:332
179732690,3sleep30-21swapper/323:25:593
179732690,3sleep30-21swapper/323:25:593
122732690,4sleep20-21swapper/200:34:462
122732690,4sleep20-21swapper/200:34:462
42412680,3chrt0-21swapper/220:00:582
42412680,3chrt0-21swapper/220:00:582
231312680,3sleep30-21swapper/321:40:413
231312680,3sleep30-21swapper/321:40:413
101802670,3sleep20-21swapper/222:40:562
58332660,3sleep20-21swapper/223:49:172
159172650,3chrt0-21swapper/300:38:413
159172650,3chrt0-21swapper/300:38:413
181032640,3chrt0-21swapper/321:34:263
182742630,4chrt0-21swapper/121:34:401
71802620,3sleep10-21swapper/121:59:521
71802620,3sleep10-21swapper/121:59:521
97032600,3chrt0-21swapper/220:15:572
44642600,3sleep30-21swapper/322:34:343
44642600,3sleep30-21swapper/322:34:343
14772590,2chrt0-21swapper/222:30:562
14772590,2chrt0-21swapper/222:30:562
325802580,3sleep2271ksoftirqd/223:06:002
325802580,3sleep2271ksoftirqd/223:06:002
126152570,4sleep20-21swapper/221:27:182
177002560,3sleep20-21swapper/219:10:522
177002560,3sleep20-21swapper/219:10:522
1901499515,8cyclictest131rcu_preempt20:25:522
1901499515,8cyclictest131rcu_preempt20:25:522
1901299505,33cyclictest6387-21ssh21:20:541
1901299505,33cyclictest6387-21ssh21:20:541
19012995024,21cyclictest21-21ksoftirqd/121:15:521
19012995024,21cyclictest21-21ksoftirqd/121:15:521
132422500,4chrt0-21swapper/123:56:311
132422500,4chrt0-21swapper/123:56:311
98442490,4sleep30-21swapper/322:40:533
222292490,3sleep30-21swapper/320:50:463
126782490,3chrt0-21swapper/121:27:251
91022480,3sleep20-21swapper/222:40:122
91022480,3sleep20-21swapper/222:40:122
1901499484,7cyclictest131rcu_preempt21:10:502
1901299481,31cyclictest2467-21ssh23:45:501
209372470,2sleep30-21swapper/322:15:543
209372470,2sleep30-21swapper/322:15:543
1901299471,38cyclictest4123-21rm23:10:521
1901299471,38cyclictest4123-21rm23:10:521
292082460,4sleep30-21swapper/321:46:283
226182460,2sleep022597-21ssh23:31:020
226182460,2sleep022597-21ssh23:31:020
1901499461,32cyclictest16550-21diskmemload22:10:562
1901499461,32cyclictest16550-21diskmemload22:10:562
1901299464,8cyclictest131rcu_preempt19:25:511
1901299464,8cyclictest131rcu_preempt19:25:511
1901199462,33cyclictest11687-21ssh23:55:530
1901199462,33cyclictest11687-21ssh23:55:530
1901199452,2cyclictest131rcu_preempt21:00:520
1901199452,2cyclictest131rcu_preempt21:00:520
185112450,2sleep20-21swapper/220:40:442
185112450,2sleep20-21swapper/220:40:442
214752440,2chrt0-21swapper/122:53:211
214752440,2chrt0-21swapper/122:53:211
1901499444,31cyclictest23240-21ssh22:55:542
1901499444,31cyclictest23240-21ssh22:55:542
19014994411,23cyclictest0-21swapper/219:20:512
19014994411,23cyclictest0-21swapper/219:20:512
1901199442,35cyclictest29306-21ssh22:25:540
1901199442,35cyclictest29306-21ssh22:25:540
283612430,3sleep00-21swapper/022:24:350
283612430,3sleep00-21swapper/022:24:350
1901199425,25cyclictest38850irq/55-eth023:40:510
1901199425,25cyclictest38850irq/55-eth023:40:510
92702410,2sleep20-21swapper/223:16:012
92702410,2sleep20-21swapper/223:16:012
278762410,10sleep333-21ksoftirqd/319:36:043
277182410,3sleep1201rcuc/123:00:531
277182410,3sleep1201rcuc/123:00:531
1901199413,22cyclictest38850irq/55-eth000:20:520
1901199413,22cyclictest38850irq/55-eth000:20:520
1901499407,20cyclictest12210-21ssh22:05:502
1901499407,20cyclictest12210-21ssh22:05:502
1901499401,32cyclictest16297-21sh00:00:522
1901499401,32cyclictest16297-21sh00:00:522
1901199402,28cyclictest1105-21sh22:30:520
1901199402,28cyclictest1105-21sh22:30:520
93722390,2sleep121-21ksoftirqd/122:01:171
93722390,2sleep121-21ksoftirqd/122:01:171
276132390,2sleep30-21swapper/322:23:173
276132390,2sleep30-21swapper/322:23:173
96382380,3sleep10-21swapper/100:30:591
96382380,3sleep10-21swapper/100:30:591
257722380,3sleep10-21swapper/122:58:081
257722380,3sleep10-21swapper/122:58:081
1901499383,17cyclictest20575-21ssh00:05:522
1901499383,17cyclictest20575-21ssh00:05:522
1901499381,3cyclictest131rcu_preempt22:20:532
1901499381,3cyclictest131rcu_preempt22:20:532
1901199385,22cyclictest38850irq/55-eth021:20:540
1901199385,22cyclictest38850irq/55-eth021:20:540
19014993611,5cyclictest21284-21grep22:15:572
19014993611,5cyclictest21284-21grep22:15:572
1901299367,22cyclictest21-21ksoftirqd/100:05:511
1901299367,22cyclictest21-21ksoftirqd/100:05:511
74762350,3sleep20-21swapper/223:14:282
74762350,3sleep20-21swapper/223:14:282
1901499351,13cyclictest19147-21sh21:35:532
1901499351,13cyclictest19147-21sh21:35:532
19011993521,10cyclictest0-21swapper/020:50:520
19011993521,10cyclictest0-21swapper/020:50:520
86412340,2sleep28652-21ssh00:30:372
86412340,2sleep28652-21ssh00:30:372
69852340,3sleep00-21swapper/023:50:470
46172340,2sleep30-21swapper/321:18:193
46172340,2sleep30-21swapper/321:18:193
38992340,3chrt0-21swapper/320:00:553
38992340,3chrt0-21swapper/320:00:553
215232340,4sleep10-21swapper/121:38:061
215232340,4sleep10-21swapper/121:38:061
19011993424,7cyclictest0-21swapper/019:25:510
19011993424,7cyclictest0-21swapper/019:25:510
177052340,3chrt0-21swapper/320:36:033
177052340,3chrt0-21swapper/320:36:033
173892340,4sleep30-21swapper/322:11:073
173892340,4sleep30-21swapper/322:11:073
157852340,2sleep119010-21cyclictest00:38:271
157852340,2sleep119010-21cyclictest00:38:271
4562330,3sleep30-21swapper/322:30:123
4562330,3sleep30-21swapper/322:30:123
209582330,3sleep20-21swapper/220:45:592
1901199331,4cyclictest0-21swapper/019:45:500
1901199331,4cyclictest0-21swapper/019:45:500
325362320,3sleep30-21swapper/323:42:263
325362320,3sleep30-21swapper/323:42:263
293612320,2sleep10-21swapper/123:39:311
1901499322,13cyclictest28906-21ssh00:15:542
129222320,3chrt12174-21/usr/sbin/munin22:05:591
129222320,3chrt12174-21/usr/sbin/munin22:05:591
115852320,3sleep20-21swapper/223:55:482
115852320,3sleep20-21swapper/223:55:482
309082310,3sleep00-21swapper/023:04:170
309082310,3sleep00-21swapper/023:04:170
278042310,3sleep20-21swapper/223:36:552
24752310,2sleep12485-21ssh00:22:081
24752310,2sleep12485-21ssh00:22:081
1901499314,10cyclictest0-21swapper/200:20:532
1901499314,10cyclictest0-21swapper/200:20:532
185792312,24sleep30-21swapper/319:11:053
185792312,24sleep30-21swapper/319:11:053
19014993011,12cyclictest19010-21cyclictest22:45:512
19014993011,12cyclictest19010-21cyclictest22:45:512
10752300,3sleep10-21swapper/122:30:511
10752300,3sleep10-21swapper/122:30:511
24352290,3chrt0-21swapper/221:15:572
24352290,3chrt0-21swapper/221:15:572
1901199293,19cyclictest18780-21sh22:50:520
1901199293,19cyclictest18780-21sh22:50:520
1901199292,19cyclictest14200-21ssh22:45:480
19011992915,3cyclictest30102-21chrt23:40:390
1901199291,23cyclictest542-21snmpd00:10:540
1901199291,23cyclictest542-21snmpd00:10:540
111672290,2sleep011177-21ssh23:55:220
111672290,2sleep011177-21ssh23:55:220
95082280,3sleep30-21swapper/300:30:573
95082280,3sleep30-21swapper/300:30:573
322472280,4sleep10-21swapper/121:50:561
322472280,4sleep10-21swapper/121:50:561
230432280,4sleep30-21swapper/320:51:023
230432280,4sleep30-21swapper/320:51:023
179242280,3sleep00-21swapper/000:01:400
179242280,3sleep00-21swapper/000:01:400
1901299272,21cyclictest21-21ksoftirqd/120:30:571
1901299272,19cyclictest5331-21latency_hist20:05:471
1901299272,19cyclictest29390-21diskmemload22:40:531
19011992719,2cyclictest0-21swapper/020:15:540
19011992715,10cyclictest11486-21ssh23:18:510
19011992715,10cyclictest11486-21ssh23:18:510
1901199271,21cyclictest14309-21taskset22:45:530
1901199271,21cyclictest14309-21taskset22:45:530
1901199271,16cyclictest131rcu_preempt22:19:550
1901199271,16cyclictest131rcu_preempt22:19:550
88552260,3sleep10-21swapper/122:39:471
88552260,3sleep10-21swapper/122:39:471
211182260,3sleep10-21swapper/120:46:021
197592260,3sleep10-21swapper/100:04:431
197592260,3sleep10-21swapper/100:04:431
1901299263,19cyclictest732-21runrttasks22:45:531
1901299263,19cyclictest732-21runrttasks22:45:531
1901199264,18cyclictest4122-21sh23:10:520
1901199264,18cyclictest4122-21sh23:10:520
1901199262,20cyclictest1914-21ssh21:15:520
1901199262,20cyclictest1914-21ssh21:15:520
1901199260,16cyclictest131rcu_preempt22:59:080
1901199260,16cyclictest131rcu_preempt22:59:080
1901299253,17cyclictest30458-21ssh23:40:531
1901299253,17cyclictest30458-21ssh23:40:531
1901199253,16cyclictest38850irq/55-eth021:40:490
1901199253,16cyclictest38850irq/55-eth021:40:490
1901199251,19cyclictest151rcuc/020:40:550
1901199251,19cyclictest151rcuc/020:40:550
19011992511,11cyclictest38850irq/55-eth023:20:540
19011992511,11cyclictest38850irq/55-eth023:20:540
1901199250,19cyclictest6302-21ntp_kernel_err20:05:580
1901199250,19cyclictest6302-21ntp_kernel_err20:05:580
305682240,3chrt0-21swapper/121:48:511
220482240,3sleep1924-1kworker/u9:123:30:551
220482240,3sleep1924-1kworker/u9:123:30:551
1901599243,12cyclictest0-21swapper/323:10:523
1901599243,12cyclictest0-21swapper/323:10:523
1901199248,10cyclictest38950irq/56-eth000:05:550
1901199248,10cyclictest38950irq/56-eth000:05:550
1901199245,3cyclictest131rcu_preempt19:55:520
1901199245,3cyclictest131rcu_preempt19:55:520
280252230,2sleep20-21swapper/200:14:412
280252230,2sleep20-21swapper/200:14:412
1901499235,13cyclictest1171-21kworker/2:021:55:522
1901499235,13cyclictest1171-21kworker/2:021:55:522
1901499233,16cyclictest18312-21grep23:26:032
1901499233,16cyclictest18312-21grep23:26:032
1901499232,18cyclictest27988-21cut21:45:552
1901299232,16cyclictest12944-21ssh23:20:541
1901299232,16cyclictest12944-21ssh23:20:541
1901199233,17cyclictest17270-21ssh23:25:520
1901199233,17cyclictest17270-21ssh23:25:520
1901199232,18cyclictest29650irq/49-vc410:36:090
1901199232,18cyclictest29650irq/49-vc410:36:090
19011992316,5cyclictest8920-21ssh00:30:470
19011992316,5cyclictest8920-21ssh00:30:470
1901199231,16cyclictest27734-21ssh21:45:530
1901199231,14cyclictest131rcu_preempt00:38:290
1901199231,14cyclictest131rcu_preempt00:38:290
19011992310,11cyclictest690-21dbus-daemon19:20:390
1901199230,15cyclictest131rcu_preempt23:08:110
1901199230,15cyclictest131rcu_preempt23:08:110
144172230,13sleep333-21ksoftirqd/323:21:243
144172230,13sleep333-21ksoftirqd/323:21:243
1901599225,14cyclictest0-21swapper/300:10:533
1901599225,14cyclictest0-21swapper/300:10:533
1901499221,18cyclictest14897-21find20:30:562
1901499221,15cyclictest5688-21find20:05:502
1901499221,15cyclictest5688-21find20:05:502
1901299221,18cyclictest9819-21ls20:15:581
1901299221,18cyclictest8704-21grep23:15:541
1901299221,18cyclictest8704-21grep23:15:541
1901299221,15cyclictest29821-21ssh00:16:051
1901199229,7cyclictest0-21swapper/022:05:520
1901199229,7cyclictest0-21swapper/022:05:520
1901199229,6cyclictest8139-21ssh22:38:370
1901199229,6cyclictest8139-21ssh22:38:370
1901199226,10cyclictest12-21ksoftirqd/021:25:500
1901199223,13cyclictest12-21ksoftirqd/019:20:500
1901199223,13cyclictest12-21ksoftirqd/019:20:500
177272226,7sleep00-21swapper/019:10:540
177272226,7sleep00-21swapper/019:10:540
87462210,3sleep30-21swapper/300:30:413
87462210,3sleep30-21swapper/300:30:413
258252210,3sleep20-21swapper/219:31:002
258252210,3sleep20-21swapper/219:31:002
1901599213,13cyclictest0-21swapper/321:50:533
1901599213,13cyclictest0-21swapper/321:50:533
1901499217,4cyclictest131rcu_preempt23:21:062
1901499217,4cyclictest131rcu_preempt23:21:062
1901499211,5cyclictest16440-21ssh21:31:382
19014992111,8cyclictest22301-21apt-get20:50:522
19014992111,8cyclictest22301-21apt-get20:50:522
1901499211,14cyclictest7850-21ls20:10:582
19014992110,9cyclictest6366-21apt-get21:20:532
19014992110,9cyclictest6366-21apt-get21:20:532
19014992110,9cyclictest32248-21sh21:50:562
19014992110,9cyclictest32248-21sh21:50:562
1901299219,9cyclictest21-21ksoftirqd/120:55:581
1901299219,9cyclictest21-21ksoftirqd/120:55:581
19012992114,4cyclictest21-21ksoftirqd/121:41:481
19012992114,4cyclictest21-21ksoftirqd/121:41:481
1901199211,2cyclictest131rcu_preempt22:02:110
1901199211,2cyclictest131rcu_preempt22:02:110
1901199211,18cyclictest13499-21ls20:25:580
1901199211,18cyclictest13499-21ls20:25:580
1901199211,14cyclictest0-21swapper/022:14:450
1901199211,14cyclictest0-21swapper/022:14:450
1901199211,13cyclictest38850irq/55-eth020:30:560
1901199210,2cyclictest131rcu_preempt21:10:580
104562210,3chrt0-21swapper/320:16:063
1901599206,8cyclictest0-21swapper/323:30:523
1901599206,8cyclictest0-21swapper/323:30:523
1901599205,12cyclictest0-21swapper/321:40:533
1901599205,12cyclictest0-21swapper/321:40:533
1901599203,13cyclictest0-21swapper/322:50:533
1901599203,13cyclictest0-21swapper/322:50:533
1901499207,10cyclictest19132-21tr22:50:562
1901499207,10cyclictest19132-21tr22:50:562
1901299209,3cyclictest131rcu_preempt19:50:471
1901299209,3cyclictest131rcu_preempt19:50:471
1901299202,15cyclictest201rcuc/119:30:511
1901299202,15cyclictest201rcuc/119:30:511
19012992015,4cyclictest21-21ksoftirqd/122:10:571
19012992015,4cyclictest21-21ksoftirqd/122:10:571
19012992012,6cyclictest28358-21sh22:24:351
19012992012,6cyclictest28358-21sh22:24:351
1901299201,17cyclictest27000-21ls21:01:041
1901299201,17cyclictest27000-21ls21:01:041
1901299201,14cyclictest29705-21apt-get21:10:501
1901199208,10cyclictest22216-21date20:50:460
1901199207,7cyclictest542-21snmpd21:57:500
1901199207,7cyclictest542-21snmpd21:57:500
1901199206,9cyclictest12-21ksoftirqd/020:35:520
1901199206,9cyclictest12-21ksoftirqd/020:35:520
1901199205,12cyclictest0-21swapper/021:30:550
1901199201,16cyclictest9007-21apt-get00:30:520
1901199201,16cyclictest9007-21apt-get00:30:520
101932200,3sleep30-21swapper/323:16:463
101932200,3sleep30-21swapper/323:16:463
82412191,3sleep30-21swapper/320:11:053
1901599195,9cyclictest0-21swapper/323:05:553
1901599195,9cyclictest0-21swapper/323:05:553
1901599195,11cyclictest0-21swapper/322:45:523
1901599195,11cyclictest0-21swapper/322:45:523
1901599191,14cyclictest0-21swapper/323:46:023
1901499197,10cyclictest12212-21load23:55:582
1901499197,10cyclictest12212-21load23:55:582
1901499191,9cyclictest18773-21fschecks_count20:40:562
1901499191,9cyclictest18773-21fschecks_count20:40:562
1901499191,4cyclictest0-21swapper/219:45:532
1901499191,4cyclictest0-21swapper/219:45:532
19014991912,5cyclictest15786-21ssh00:38:272
19014991912,5cyclictest15786-21ssh00:38:272
1901499191,15cyclictest0-21swapper/221:00:522
1901499191,15cyclictest0-21swapper/221:00:522
1901499191,10cyclictest20588-21cat19:16:032
1901499190,15cyclictest24295-21timerandwakeup19:26:032
1901499190,15cyclictest24295-21timerandwakeup19:26:032
1901299196,7cyclictest21-21ksoftirqd/122:25:531
1901299196,7cyclictest21-21ksoftirqd/122:25:531
19012991914,3cyclictest21-21ksoftirqd/122:19:211
19012991914,3cyclictest21-21ksoftirqd/122:19:211
1901299191,16cyclictest7072-21ssh23:50:511
1901299191,16cyclictest7072-21ssh23:50:511
1901299191,15cyclictest32082-21taskset23:05:551
1901299191,15cyclictest32082-21taskset23:05:551
1901299191,11cyclictest732-21runrttasks20:26:371
1901299191,11cyclictest732-21runrttasks20:26:371
1901199199,5cyclictest12-21ksoftirqd/000:16:230
1901199197,10cyclictest38950irq/56-eth021:40:560
1901199197,10cyclictest38950irq/56-eth021:40:560
1901199196,5cyclictest29009-21fschecks_count19:40:510
1901199192,13cyclictest12-21ksoftirqd/019:30:510
1901199192,13cyclictest12-21ksoftirqd/019:30:510
1901199191,16cyclictest27992-21cut21:05:540
1901199191,16cyclictest27992-21cut21:05:540
1901199191,10cyclictest732-21runrttasks19:52:510
1901199191,10cyclictest732-21runrttasks19:52:510
19011991910,6cyclictest5528-21date20:05:420
19011991910,6cyclictest5528-21date20:05:420
19011991910,6cyclictest0-21swapper/020:25:430
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional