You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-20 - 07:18

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #4, slot #1

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1s.osadl.org (updated Wed May 20, 2026 00:44:17)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1355121040,2chrt13548-21bingrep20:47:460
225552910,3chrt0-21swapper/222:37:512
225552910,3chrt0-21swapper/222:37:512
42132880,3sleep20-21swapper/220:22:482
42132880,3sleep20-21swapper/220:22:482
24082710,3sleep20-21swapper/200:37:472
24082710,3sleep20-21swapper/200:37:472
168882630,3sleep2351rcuc/222:22:522
168882630,3sleep2351rcuc/222:22:522
259792620,3chrt0-21swapper/322:47:453
259792620,3chrt0-21swapper/322:47:453
9724996150,4cyclictest0-21swapper/000:02:340
9724996150,4cyclictest0-21swapper/000:02:340
296522610,2chrt29654-21date21:32:360
296522610,2chrt29654-21date21:32:360
224352560,3chrt0-21swapper/300:03:153
224352560,3chrt0-21swapper/300:03:153
182362550,3sleep018228-21rt-features23:52:500
182362550,3sleep018228-21rt-features23:52:500
301542540,2sleep00-21swapper/021:32:440
301542540,2sleep00-21swapper/021:32:440
215392520,3sleep30-21swapper/319:42:453
215392520,3sleep30-21swapper/319:42:453
972599481,6cyclictest24177-21taskset19:48:411
972599481,6cyclictest24177-21taskset19:48:411
972599461,8cyclictest6874-21taskset20:30:331
972599461,8cyclictest6874-21taskset20:30:331
972599461,7cyclictest11204-21taskset23:33:321
972599461,7cyclictest11204-21taskset23:33:321
972599451,8cyclictest31371-21seq21:37:271
972599451,8cyclictest31371-21seq21:37:271
972599451,1cyclictest0-21swapper/122:43:071
972599451,1cyclictest0-21swapper/122:43:071
972599451,1cyclictest0-21swapper/122:43:071
75882450,3chrt0-21swapper/221:57:512
75882450,3chrt0-21swapper/221:57:512
317372450,2chrt0-21swapper/321:37:423
317372450,2chrt0-21swapper/321:37:423
217892450,3sleep20-21swapper/221:09:272
217892450,3sleep20-21swapper/221:09:272
173412450,5sleep1281ktimers/122:22:551
173412450,5sleep1281ktimers/122:22:551
972599440,8cyclictest16185-21seq20:54:291
972599440,8cyclictest16185-21seq20:54:291
255602440,2sleep025737-21grep22:47:430
255602440,2sleep025737-21grep22:47:430
197372440,3sleep30-21swapper/321:02:533
197372440,3sleep30-21swapper/321:02:533
197372440,3sleep30-21swapper/321:02:533
171632440,6sleep017166-21idleruntime-cro23:52:300
171632440,6sleep017166-21idleruntime-cro23:52:300
171632440,6sleep017166-21idleruntime-cro23:52:300
149272440,2sleep30-21swapper/322:17:503
149272440,2sleep30-21swapper/322:17:503
972599421,8cyclictest4275-21taskset21:48:471
972599421,8cyclictest4275-21taskset21:48:471
972599421,6cyclictest15674-21taskset22:20:121
972599421,6cyclictest15674-21taskset22:20:121
972599421,4cyclictest1183-21seq20:15:061
972599421,4cyclictest1183-21seq20:15:061
972599421,1cyclictest0-21swapper/120:46:031
972599421,1cyclictest0-21swapper/120:46:031
972599421,1cyclictest0-21swapper/120:40:331
972599421,1cyclictest0-21swapper/120:40:331
287242420,3sleep30-21swapper/320:02:433
287242420,3sleep30-21swapper/320:02:433
239412420,3chrt0-21swapper/022:42:430
239412420,3chrt0-21swapper/022:42:430
239412420,3chrt0-21swapper/022:42:430
972599411,3cyclictest0-21swapper/120:10:031
972599411,3cyclictest0-21swapper/120:10:031
288792410,2chrt28882-21if_eth020:02:440
288792410,2chrt28882-21if_eth020:02:440
972599401,3cyclictest0-21swapper/123:44:091
972599401,3cyclictest0-21swapper/123:44:091
972599401,3cyclictest0-21swapper/123:44:091
972599401,1cyclictest0-21swapper/122:32:221
972599401,1cyclictest0-21swapper/122:32:221
972599401,1cyclictest0-21swapper/100:41:401
972599401,1cyclictest0-21swapper/100:41:401
972599401,1cyclictest0-21swapper/100:36:101
972599401,1cyclictest0-21swapper/100:36:101
191992400,3sleep30-21swapper/319:37:423
191992400,3sleep30-21swapper/319:37:423
972599391,3cyclictest0-21swapper/121:40:051
972599391,3cyclictest0-21swapper/121:40:051
972599391,3cyclictest0-21swapper/120:27:331
972599391,3cyclictest0-21swapper/120:27:331
972599391,3cyclictest0-21swapper/119:30:011
972599391,3cyclictest0-21swapper/119:30:011
972599391,2cyclictest0-21swapper/122:40:371
972599391,2cyclictest0-21swapper/122:40:371
972599391,2cyclictest0-21swapper/121:45:051
972599391,2cyclictest0-21swapper/121:45:051
972599391,2cyclictest0-21swapper/121:45:051
972599391,1cyclictest0-21swapper/123:11:071
972599391,1cyclictest0-21swapper/123:11:071
972599391,1cyclictest0-21swapper/121:58:051
972599391,1cyclictest0-21swapper/121:58:051
972599391,1cyclictest0-21swapper/121:53:061
972599391,1cyclictest0-21swapper/121:53:061
972599391,1cyclictest0-21swapper/121:53:061
972599390,5cyclictest20935-40mandb00:02:411
972599390,5cyclictest20935-40mandb00:02:411
9724993926,10cyclictest21133-21/usr/sbin/munin00:02:460
9724993926,10cyclictest21133-21/usr/sbin/munin00:02:460
268032390,3sleep20-21swapper/200:17:412
268032390,3sleep20-21swapper/200:17:412
13902390,3chrt0-21swapper/223:07:502
13902390,3chrt0-21swapper/223:07:502
972599389,20cyclictest749-21runrttasks19:34:491
972599389,20cyclictest749-21runrttasks19:34:491
972599381,8cyclictest9335-21ntp_states22:02:511
972599381,8cyclictest9335-21ntp_states22:02:511
972599381,8cyclictest9335-21ntp_states22:02:511
9725993814,21cyclictest0-21swapper/120:47:521
972599381,3cyclictest0-21swapper/122:10:061
972599381,3cyclictest0-21swapper/122:10:061
972599381,3cyclictest0-21swapper/122:10:061
972599381,3cyclictest0-21swapper/121:28:051
972599381,3cyclictest0-21swapper/121:28:051
972599381,3cyclictest0-21swapper/121:17:041
972599381,3cyclictest0-21swapper/120:07:021
972599381,3cyclictest0-21swapper/120:07:021
972599381,3cyclictest0-21swapper/120:01:021
972599381,3cyclictest0-21swapper/120:01:021
972599381,2cyclictest0-21swapper/123:39:091
972599381,2cyclictest0-21swapper/123:39:091
972599381,2cyclictest0-21swapper/123:26:381
972599381,2cyclictest0-21swapper/123:26:381
972599381,1cyclictest0-21swapper/123:07:231
972599381,1cyclictest0-21swapper/123:07:231
972599381,1cyclictest0-21swapper/122:59:371
972599381,1cyclictest0-21swapper/122:59:371
972599381,1cyclictest0-21swapper/121:12:041
972599381,1cyclictest0-21swapper/121:12:041
972599381,1cyclictest0-21swapper/121:04:341
972599381,1cyclictest0-21swapper/121:04:341
972599381,1cyclictest0-21swapper/121:04:341
972599381,1cyclictest0-21swapper/119:25:011
972599381,1cyclictest0-21swapper/119:25:011
972599381,1cyclictest0-21swapper/100:30:101
972599381,1cyclictest0-21swapper/100:30:101
9724993828,7cyclictest1771-21fschecks_count20:17:430
9724993828,7cyclictest1771-21fschecks_count20:17:430
162652380,2sleep30-21swapper/323:47:513
162652380,2sleep30-21swapper/323:47:513
162652380,2sleep30-21swapper/323:47:513
972599371,8cyclictest8879-21sshd20:37:261
972599371,8cyclictest8879-21sshd20:37:261
972599371,3cyclictest0-21swapper/123:48:091
972599371,3cyclictest0-21swapper/123:48:091
972599371,3cyclictest0-21swapper/123:48:091
972599371,3cyclictest0-21swapper/121:21:341
972599371,3cyclictest0-21swapper/121:21:341
972599371,3cyclictest0-21swapper/119:45:321
972599371,3cyclictest0-21swapper/119:45:321
972599371,2cyclictest0-21swapper/120:22:181
972599371,2cyclictest0-21swapper/120:22:181
972599371,1cyclictest0-21swapper/119:20:511
972599371,1cyclictest0-21swapper/100:26:101
972599371,1cyclictest0-21swapper/100:26:101
972599371,11cyclictest25443-21ntp_states19:52:501
972599371,11cyclictest25443-21ntp_states19:52:501
204312370,4sleep3431rcuc/322:32:453
204312370,4sleep3431rcuc/322:32:453
204312370,4sleep3431rcuc/322:32:453
972599361,3cyclictest0-21swapper/122:53:071
972599361,3cyclictest0-21swapper/122:53:071
972599361,3cyclictest0-21swapper/121:25:341
972599361,3cyclictest0-21swapper/121:25:341
972599361,3cyclictest0-21swapper/121:00:341
972599361,3cyclictest0-21swapper/121:00:341
972599361,3cyclictest0-21swapper/100:16:401
972599361,3cyclictest0-21swapper/100:16:401
972599361,2cyclictest0-21swapper/123:57:091
972599361,2cyclictest0-21swapper/123:57:091
972599361,2cyclictest0-21swapper/100:10:391
972599361,2cyclictest0-21swapper/100:10:391
972599361,2cyclictest0-21swapper/100:02:091
972599361,2cyclictest0-21swapper/100:02:091
972599361,1cyclictest0-21swapper/123:29:381
972599361,1cyclictest0-21swapper/123:29:381
972599361,1cyclictest0-21swapper/123:21:381
972599361,1cyclictest0-21swapper/123:21:381
972599361,1cyclictest0-21swapper/123:21:381
972599361,1cyclictest0-21swapper/122:16:091
972599361,1cyclictest0-21swapper/122:16:091
972599361,10cyclictest27373-21df22:52:411
972599361,10cyclictest27373-21df22:52:411
972599361,10cyclictest20314-21/usr/sbin/munin22:32:451
972599361,10cyclictest20314-21/usr/sbin/munin22:32:451
972599361,10cyclictest20314-21/usr/sbin/munin22:32:451
226782360,1chrt0-21swapper/019:47:320
226782360,1chrt0-21swapper/019:47:320
972599351,8cyclictest20413-21sshd19:38:271
972599351,8cyclictest20413-21sshd19:38:271
972599351,1cyclictest0-21swapper/100:20:091
972599351,1cyclictest0-21swapper/100:20:091
263862350,2sleep00-21swapper/021:22:440
263862350,2sleep00-21swapper/021:22:440
972899341,22cyclictest21019-21chrt00:02:343
972899341,22cyclictest21019-21chrt00:02:343
972599341,10cyclictest2825-21irqstats23:12:441
972599341,10cyclictest2825-21irqstats23:12:441
9724993223,7cyclictest28856-21fschecks_time00:22:450
9724993223,7cyclictest28856-21fschecks_time00:22:450
13392320,3sleep20-21swapper/221:42:432
13392320,3sleep20-21swapper/221:42:432
13392320,3sleep20-21swapper/221:42:432
9724993127,3cyclictest6462-21timerwakeupswit20:27:520
9724993127,3cyclictest6462-21timerwakeupswit20:27:520
9724993123,6cyclictest26203-21sshd00:13:510
9724993123,6cyclictest26203-21sshd00:13:510
35602310,1sleep30-21swapper/320:22:413
35602310,1sleep30-21swapper/320:22:413
172982310,3sleep30-21swapper/320:57:463
172982310,3sleep30-21swapper/320:57:463
9724993026,3cyclictest18454-21kernelversion22:27:450
9724993026,3cyclictest18454-21kernelversion22:27:450
9724993023,5cyclictest28853-21sshd22:56:340
9724993023,5cyclictest28853-21sshd22:56:340
9724993023,5cyclictest12075-21kernelversion19:17:460
9724992922,5cyclictest13997-21kernelversion19:22:440
9724992922,5cyclictest13997-21kernelversion19:22:440
9724992920,7cyclictest3739-21cut20:22:420
9724992920,7cyclictest3739-21cut20:22:420
972699283,14cyclictest209932sleep200:02:342
972699283,14cyclictest209932sleep200:02:342
9724992823,3cyclictest0-21swapper/023:02:300
9724992823,3cyclictest0-21swapper/023:02:300
9724992822,5cyclictest1262-21fschecks_time21:42:430
9724992822,5cyclictest1262-21fschecks_time21:42:430
9724992822,5cyclictest1262-21fschecks_time21:42:430
9724992819,7cyclictest2500-21fschecks_count23:12:410
9724992819,7cyclictest2500-21fschecks_count23:12:410
167572280,2sleep20-21swapper/220:57:412
167572280,2sleep20-21swapper/220:57:412
9728992721,4cyclictest259352sleep319:52:543
9728992721,4cyclictest259352sleep319:52:543
9724992721,4cyclictest0-21swapper/020:37:450
9724992721,4cyclictest0-21swapper/020:37:450
9724992720,5cyclictest9922-21cpuspeed23:32:410
9724992720,5cyclictest9922-21cpuspeed23:32:410
9724992720,5cyclictest22759-21kernelversion21:12:450
9724992621,3cyclictest550-21cron21:17:390
9724992621,3cyclictest550-21cron21:17:390
9724992621,3cyclictest22214-21latency22:37:450
9724992621,3cyclictest22214-21latency22:37:450
9724992621,3cyclictest0-21swapper/000:37:440
9724992621,3cyclictest0-21swapper/000:37:440
9724992620,4cyclictest12261-21/usr/sbin/munin22:12:410
9724992620,4cyclictest12261-21/usr/sbin/munin22:12:410
9724992620,4cyclictest12261-21/usr/sbin/munin22:12:410
9724992620,4cyclictest0-21swapper/023:07:410
9724992620,4cyclictest0-21swapper/023:07:410
9724992619,5cyclictest3798-21wc21:47:530
9724992619,5cyclictest3798-21wc21:47:530
9724992619,5cyclictest16105-21sshd20:53:270
9724992619,5cyclictest16105-21sshd20:53:270
9724992619,5cyclictest1543-21latency_hist00:37:300
9724992619,5cyclictest1543-21latency_hist00:37:300
9724992618,6cyclictest5036-21taskset21:52:430
9724992618,6cyclictest5036-21taskset21:52:430
9724992618,6cyclictest5036-21taskset21:52:430
9724992520,3cyclictest0-21swapper/022:37:300
9724992520,3cyclictest0-21swapper/022:37:300
9724992520,3cyclictest0-21swapper/022:37:300
9724992519,5cyclictest18238-21threads19:32:500
9724992519,5cyclictest18238-21threads19:32:500
9724992519,4cyclictest18303-21latency_hist21:02:300
9724992519,4cyclictest18303-21latency_hist21:02:300
9724992517,6cyclictest0-21swapper/023:12:480
9724992517,6cyclictest0-21swapper/023:12:480
78442250,2chrt7845-21missed_timers20:32:470
78442250,2chrt7845-21missed_timers20:32:470
324172250,4sleep032415-21df00:32:420
324172250,4sleep032415-21df00:32:420
250052250,4sleep30-21swapper/300:12:433
250052250,4sleep30-21swapper/300:12:433
9724992419,4cyclictest6295-21fschecks_count23:22:430
9724992419,4cyclictest6295-21fschecks_count23:22:430
9724992419,4cyclictest15851-21cron22:22:300
9724992419,4cyclictest15851-21cron22:22:300
9724992418,5cyclictest22096-21sh21:12:320
9724992418,5cyclictest22096-21sh21:12:320
9724992418,5cyclictest17735-21latency_hist22:27:410
9724992417,5cyclictest31625-21sshd20:08:260
9724992417,5cyclictest31625-21sshd20:08:260
9724992417,5cyclictest19026-21kernelversion21:02:450
9724992417,5cyclictest19026-21kernelversion21:02:450
9724992417,5cyclictest19026-21kernelversion21:02:450
9724992417,5cyclictest13623-21/usr/sbin/munin23:42:470
9724992417,5cyclictest13623-21/usr/sbin/munin23:42:470
9724992417,5cyclictest13623-21/usr/sbin/munin23:42:470
9724992318,4cyclictest595-21sshd20:46:270
9724992318,4cyclictest595-21sshd20:46:270
9724992315,6cyclictest821-21ls20:12:530
9724992315,6cyclictest821-21ls20:12:530
9724992218,2cyclictest0-21swapper/019:27:460
9724992218,2cyclictest0-21swapper/019:27:460
9724992218,2cyclictest0-21swapper/000:07:570
9724992218,2cyclictest0-21swapper/000:07:570
9724992217,3cyclictest0-21swapper/023:17:540
9724992217,3cyclictest0-21swapper/023:17:540
9724992217,3cyclictest0-21swapper/023:17:540
9724992217,3cyclictest0-21swapper/021:42:320
9724992217,3cyclictest0-21swapper/021:42:320
9724992216,4cyclictest11564-21latency_hist23:37:320
9724992216,4cyclictest11564-21latency_hist23:37:320
9724992216,4cyclictest0-21swapper/019:37:430
9724992216,4cyclictest0-21swapper/019:37:430
9724992216,3cyclictest0-21swapper/022:22:530
9724992216,3cyclictest0-21swapper/022:12:500
9724992216,3cyclictest0-21swapper/022:12:500
9724992216,3cyclictest0-21swapper/022:07:310
9724992216,3cyclictest0-21swapper/022:07:310
9724992216,3cyclictest0-21swapper/022:07:310
236752220,3sleep20-21swapper/222:42:412
236752220,3sleep20-21swapper/222:42:412
236752220,3sleep20-21swapper/222:42:412
206252220,3sleep30-21swapper/321:07:433
206252220,3sleep30-21swapper/321:07:433
9724992117,2cyclictest0-21swapper/019:57:530
9724992117,2cyclictest0-21swapper/019:57:530
9724992115,4cyclictest23119-21fw_packets19:47:430
9724992115,4cyclictest23119-21fw_packets19:47:430
9724992114,4cyclictest0-21swapper/023:37:520
9724992114,4cyclictest0-21swapper/023:37:520
163352210,3chrt0-21swapper/322:22:443
163352210,3chrt0-21swapper/322:22:443
972699203,10cyclictest0-21swapper/220:28:112
972699203,10cyclictest0-21swapper/220:28:112
9724992013,5cyclictest25554-21rt-features19:52:510
9724992013,5cyclictest25554-21rt-features19:52:510
972499201,16cyclictest27091-21ls00:17:430
972499201,16cyclictest27091-21ls00:17:430
972699193,9cyclictest0-21swapper/223:13:332
972699193,9cyclictest0-21swapper/223:13:332
972699193,10cyclictest0-21swapper/220:03:062
972699193,10cyclictest0-21swapper/220:03:062
9728991812,4cyclictest237522sleep322:42:423
9728991812,4cyclictest237522sleep322:42:423
9728991812,4cyclictest237522sleep322:42:423
972899181,12cyclictest8672-21chrt22:02:423
972899181,12cyclictest8672-21chrt22:02:423
972899181,12cyclictest8672-21chrt22:02:423
972699183,9cyclictest0-21swapper/221:53:322
972699183,9cyclictest0-21swapper/221:53:322
972699183,9cyclictest0-21swapper/221:53:322
972699183,9cyclictest0-21swapper/221:49:322
972699183,9cyclictest0-21swapper/221:49:322
972699183,9cyclictest0-21swapper/219:23:002
972699183,9cyclictest0-21swapper/219:23:002
972699182,11cyclictest12517-21taskset22:12:422
972699182,11cyclictest12517-21taskset22:12:422
972699181,12cyclictest8382-21taskset20:32:532
972699181,12cyclictest8382-21taskset20:32:532
972699181,12cyclictest14948-21taskset20:52:432
972699181,12cyclictest14948-21taskset20:52:432
9724991810,6cyclictest7697-21cat21:57:520
9724991810,6cyclictest7697-21cat21:57:520
56742180,1sleep3431rcuc/320:27:433
56742180,1sleep3431rcuc/320:27:433
54942180,3sleep30-21swapper/323:17:563
54942180,3sleep30-21swapper/323:17:563
54942180,3sleep30-21swapper/323:17:563
972899171,11cyclictest24053-21chrt21:17:363
972899171,11cyclictest1500-21taskset21:42:443
972899171,11cyclictest1500-21taskset21:42:443
972899171,11cyclictest1500-21taskset21:42:443
972699172,11cyclictest0-21swapper/200:32:542
972699172,11cyclictest0-21swapper/200:32:542
972699171,12cyclictest30642-21chrt20:07:442
972699171,12cyclictest30642-21chrt20:07:442
972699171,12cyclictest0-21swapper/223:42:452
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional