You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-22 - 11:56

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #4, slot #1

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1s.osadl.org (updated Fri May 22, 2026 00:44:17)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
230392990,3sleep30-21swapper/321:07:333
230392990,3sleep30-21swapper/321:07:333
206682970,4chrt0-21swapper/322:27:363
206682970,4chrt0-21swapper/322:27:363
212642940,4chrt21268-21unixbench_multi19:32:550
57422920,2sleep20-21swapper/223:12:462
57422920,2sleep20-21swapper/223:12:462
253012660,2sleep30-21swapper/300:03:403
253012660,2sleep30-21swapper/300:03:403
253012660,2sleep30-21swapper/300:03:403
110432630,3sleep20-21swapper/223:27:452
110432630,3sleep20-21swapper/223:27:452
30612610,3sleep30-21swapper/321:37:563
30612610,3sleep30-21swapper/321:37:563
30612610,3sleep30-21swapper/321:37:563
95082600,3sleep09507-21cat21:57:420
95082600,3sleep09507-21cat21:57:420
95082600,3sleep09507-21cat21:57:420
182202590,3sleep2351rcuc/222:17:552
182202590,3sleep2351rcuc/222:17:552
182202590,3sleep2351rcuc/222:17:552
161632580,3chrt0-21swapper/322:12:533
161632580,3chrt0-21swapper/322:12:533
69932540,3sleep30-21swapper/320:22:463
69932540,3sleep30-21swapper/320:22:463
202372510,3sleep016118-21kworker/u9:3+xprtiod23:52:440
6122470,3sleep20-21swapper/220:07:382
6122470,3sleep20-21swapper/220:07:382
156832470,3sleep20-21swapper/223:37:542
156832470,3sleep20-21swapper/223:37:542
156832470,3sleep20-21swapper/223:37:542
156832470,3sleep20-21swapper/223:37:542
12682994742,3cyclictest0-21swapper/000:02:400
12682994742,3cyclictest0-21swapper/000:02:400
12682994742,3cyclictest0-21swapper/000:02:400
12682994742,3cyclictest0-21swapper/000:02:400
238962460,3sleep20-21swapper/200:02:432
238962460,3sleep20-21swapper/200:02:432
238962460,3sleep20-21swapper/200:02:432
238962460,3sleep20-21swapper/200:02:432
12682994131,7cyclictest0-21swapper/000:03:050
12682994131,7cyclictest0-21swapper/000:03:050
12682994131,7cyclictest0-21swapper/000:03:050
44212400,2chrt4423-21basename00:37:340
44212400,2chrt4423-21basename00:37:340
12683994035,3cyclictest28004-21kworker/u9:2-nfsiod00:02:341
12683994035,3cyclictest28004-21kworker/u9:2-nfsiod00:02:341
12683994035,3cyclictest28004-21kworker/u9:2-nfsiod00:02:341
12683994035,3cyclictest28004-21kworker/u9:2-nfsiod00:02:341
274592380,2sleep20-21swapper/200:12:322
274592380,2sleep20-21swapper/200:12:322
1268299379,12cyclictest15970-21chrt23:39:010
1268299379,12cyclictest15970-21chrt23:39:010
1268299379,12cyclictest15970-21chrt23:39:010
1268299379,12cyclictest15970-21chrt23:39:010
1268299378,11cyclictest30881-21chrt00:18:370
1268299378,11cyclictest30881-21chrt00:18:370
12682993732,4cyclictest572-21cups-browsed20:42:520
12682993732,4cyclictest572-21cups-browsed20:42:520
12682993730,5cyclictest14990-21latency_hist22:12:330
12682993730,5cyclictest14990-21latency_hist22:12:330
12682993711,10cyclictest5831-21awk00:37:560
12682993711,10cyclictest5831-21awk00:37:560
12682993711,10cyclictest5831-21awk00:37:560
39872360,2sleep14039-21ntpq23:07:511
39872360,2sleep14039-21ntpq23:07:511
39872360,2sleep14039-21ntpq23:07:511
1268299369,17cyclictest7980-21taskset20:27:170
1268299369,17cyclictest7980-21taskset20:27:170
12682993616,11cyclictest0-21swapper/021:27:430
12682993616,11cyclictest0-21swapper/021:27:430
214672350,2sleep020758-21/usr/sbin/munin22:27:500
214672350,2sleep020758-21/usr/sbin/munin22:27:500
207132350,4chrt20198-21/usr/sbin/munin23:52:480
207132350,4chrt20198-21/usr/sbin/munin23:52:480
207132350,4chrt20198-21/usr/sbin/munin23:52:480
12683993528,5cyclictest11194-21latency_hist22:02:321
12683993528,5cyclictest11194-21latency_hist22:02:321
12683993526,7cyclictest8461-21processes21:52:531
12683993526,7cyclictest8461-21processes21:52:531
12683993526,7cyclictest8461-21processes21:52:531
1268299359,11cyclictest7827-21awk23:17:520
1268299359,11cyclictest7827-21awk23:17:520
12682993530,3cyclictest2921-21cut20:12:440
12682993530,3cyclictest2921-21cut20:12:440
12683993427,5cyclictest25052-21acpi21:12:431
12683993427,5cyclictest25052-21acpi21:12:431
12683993427,5cyclictest25052-21acpi21:12:431
1268299349,11cyclictest31500-21cut20:02:440
1268299349,11cyclictest31500-21cut20:02:440
1268299346,11cyclictest2222-21taskset20:10:440
1268299346,11cyclictest2222-21taskset20:10:440
1268299346,11cyclictest2222-21taskset20:10:440
44492330,1sleep14532-21cat20:17:431
44492330,1sleep14532-21cat20:17:431
274642330,2sleep027595-21df19:52:410
274642330,2sleep027595-21df19:52:410
234362330,2sleep30-21swapper/321:07:453
234362330,2sleep30-21swapper/321:07:453
234362330,2sleep30-21swapper/321:07:453
12683993314,11cyclictest12681-21cyclictest20:37:521
12683993314,11cyclictest12681-21cyclictest20:37:521
1268299338,9cyclictest11690-21perl22:02:440
1268299338,9cyclictest11690-21perl22:02:440
1268299336,11cyclictest15908-21ntpq22:12:500
1268299336,11cyclictest15908-21ntpq22:12:500
12682993327,4cyclictest5871-21head23:12:470
12682993327,4cyclictest5871-21head23:12:470
12682993323,8cyclictest27978-21threads21:17:560
12682993323,8cyclictest27978-21threads21:17:560
1269099321,22cyclictest23859-21taskset00:02:403
1269099321,22cyclictest23859-21taskset00:02:403
1269099321,22cyclictest23859-21taskset00:02:403
1269099321,22cyclictest23859-21taskset00:02:403
1268299328,8cyclictest20338-21chrt22:24:320
1268299328,8cyclictest20338-21chrt22:24:320
1268299326,11cyclictest2989-21taskset23:06:150
1268299326,11cyclictest2989-21taskset23:06:150
1268299326,11cyclictest2989-21taskset23:06:150
1268299325,12cyclictest5674-21cat21:47:370
1268299325,12cyclictest5674-21cat21:47:370
1268299325,12cyclictest5674-21cat21:47:370
1268299324,11cyclictest16418-21bingrep20:47:470
1268299324,11cyclictest16418-21bingrep20:47:470
12682993222,8cyclictest0-21swapper/019:29:280
12682993222,8cyclictest0-21swapper/019:29:280
12682993222,8cyclictest0-21swapper/019:29:280
12683993119,3cyclictest617-21lldpd19:55:131
1268299316,8cyclictest29129-21seq00:15:030
1268299316,8cyclictest29129-21seq00:15:030
1268299316,13cyclictest10218-21sed20:32:420
1268299316,10cyclictest23466-21fschecks_time21:07:450
1268299316,10cyclictest23466-21fschecks_time21:07:450
1268299316,10cyclictest23466-21fschecks_time21:07:450
1268299315,13cyclictest10344-21seq23:23:570
1268299315,13cyclictest10344-21seq23:23:570
1268299315,12cyclictest29132-21taskset19:56:570
1268299315,10cyclictest13261-21wc22:07:410
1268299315,10cyclictest13261-21wc22:07:410
12682993123,5cyclictest9893-21sh20:32:430
12683993024,5cyclictest22475-21latency_hist22:32:331
12683993024,5cyclictest22475-21latency_hist22:32:331
12683993015,9cyclictest90-1pr/ttyS021:27:431
12683993014,10cyclictest90-1pr/ttyS023:47:451
12683993014,10cyclictest90-1pr/ttyS023:47:451
1268299309,9cyclictest19579-21grep23:47:550
1268299309,7cyclictest11559-21awk23:27:510
1268299309,7cyclictest11559-21awk23:27:510
1268299307,10cyclictest30939-21taskset20:01:150
1268299307,10cyclictest30939-21taskset20:01:150
1268299306,8cyclictest31598-21chrt22:55:230
1268299306,8cyclictest31598-21chrt22:55:230
1268299306,11cyclictest25916-21seq22:39:270
1268299306,11cyclictest25916-21seq22:39:270
1268299305,8cyclictest14678-21systemd-journal21:05:450
1268299305,8cyclictest14678-21systemd-journal21:05:450
1268299305,11cyclictest25578-21cat21:12:470
1268299305,11cyclictest25578-21cat21:12:470
1268299304,8cyclictest22681-21ntp_states19:37:500
1268299304,8cyclictest22681-21ntp_states19:37:500
1268299303,12cyclictest13756-21cat23:32:540
1268299303,12cyclictest13756-21cat23:32:540
12682993024,5cyclictest12657-21needreboot20:37:470
12682993024,5cyclictest12657-21needreboot20:37:470
12682993023,5cyclictest23956-21fschecks_count19:42:430
12682993023,5cyclictest23956-21fschecks_count19:42:430
12682993023,5cyclictest23956-21fschecks_count19:42:430
1269099292,21cyclictest0-21swapper/322:12:323
1269099292,21cyclictest0-21swapper/322:12:323
12683992924,4cyclictest12681-21cyclictest21:32:371
12683992924,4cyclictest12681-21cyclictest21:32:371
12683992915,3cyclictest90-1pr/ttyS023:07:431
12683992915,3cyclictest90-1pr/ttyS023:07:431
12683992912,4cyclictest90-1pr/ttyS019:22:441
12683992912,4cyclictest90-1pr/ttyS019:22:441
12683992910,10cyclictest0-21swapper/100:22:341
12683992910,10cyclictest0-21swapper/100:22:341
1268299298,7cyclictest31426-21awk21:27:500
1268299298,7cyclictest31426-21awk21:27:500
1268299295,6cyclictest31929-21irqstats00:22:470
1268299295,6cyclictest31929-21irqstats00:22:470
1268299295,6cyclictest31929-21irqstats00:22:470
1268299294,15cyclictest749-21runrttasks22:51:000
1268299294,15cyclictest749-21runrttasks22:51:000
1268299293,9cyclictest18630-21perl23:47:450
12682992924,3cyclictest749-21runrttasks21:36:400
12682992924,3cyclictest749-21runrttasks21:36:400
12682992922,5cyclictest8442-21fschecks_time20:27:440
85932280,3sleep20-21swapper/221:52:542
85932280,3sleep20-21swapper/221:52:542
85932280,3sleep20-21swapper/221:52:542
1268399286,4cyclictest181rcu_preempt22:05:441
1268399286,4cyclictest181rcu_preempt22:05:441
12683992823,4cyclictest12681-21cyclictest23:23:471
12683992823,4cyclictest12681-21cyclictest23:23:471
12683992823,3cyclictest31630-21gltestperf20:02:451
12683992823,3cyclictest31630-21gltestperf20:02:451
12683992822,4cyclictest1346-21avahi-daemon23:02:321
12683992822,4cyclictest1346-21avahi-daemon23:02:321
12683992821,5cyclictest3233-21sshd23:07:401
12683992816,4cyclictest90-1pr/ttyS021:12:531
12683992816,4cyclictest90-1pr/ttyS021:12:531
12683992813,8cyclictest90-1pr/ttyS022:22:531
12683992813,8cyclictest90-1pr/ttyS022:22:531
12683992813,4cyclictest90-1pr/ttyS023:12:511
12683992813,4cyclictest90-1pr/ttyS023:12:511
12683992810,9cyclictest0-21swapper/120:12:351
12683992810,9cyclictest0-21swapper/120:12:351
12683992810,9cyclictest0-21swapper/120:12:351
1268299286,5cyclictest6278-21irqstats21:47:460
1268299286,5cyclictest6278-21irqstats21:47:460
1268299284,10cyclictest1276-21latency_hist23:02:340
1268299284,10cyclictest1276-21latency_hist23:02:340
1268299283,11cyclictest18459-21runrttasks22:19:360
1268299283,11cyclictest18459-21runrttasks22:19:360
1268299283,11cyclictest18459-21runrttasks22:19:360
1269099271,6cyclictest1439-21chrt21:33:053
1269099271,6cyclictest1439-21chrt21:33:053
12683992721,5cyclictest2754-21ntp_offset21:37:521
12683992721,5cyclictest2754-21ntp_offset21:37:521
12683992721,5cyclictest2754-21ntp_offset21:37:521
12683992721,3cyclictest0-21swapper/123:47:321
12683992721,3cyclictest0-21swapper/123:47:321
12683992720,5cyclictest19920-21if_eth020:57:451
12683992720,5cyclictest19920-21if_eth020:57:451
12683992720,5cyclictest16435-21latency_hist23:42:391
12683992720,5cyclictest16435-21latency_hist23:42:391
12683992720,5cyclictest16435-21latency_hist23:42:391
12683992720,5cyclictest16435-21latency_hist23:42:391
12683992718,7cyclictest27855-21gltestperf19:52:431
12683992718,2cyclictest181rcu_preempt19:27:481
12683992718,2cyclictest181rcu_preempt19:27:481
12683992718,2cyclictest181rcu_preempt19:27:481
12683992716,4cyclictest90-1pr/ttyS022:12:531
12683992716,4cyclictest90-1pr/ttyS022:12:531
12683992715,3cyclictest181rcu_preempt23:52:551
12683992715,3cyclictest181rcu_preempt23:52:551
12683992715,3cyclictest181rcu_preempt23:52:551
12683992715,3cyclictest181rcu_preempt20:42:541
12683992715,3cyclictest181rcu_preempt20:42:541
12683992713,8cyclictest90-1pr/ttyS000:07:431
12683992713,8cyclictest90-1pr/ttyS000:07:431
12683992713,8cyclictest90-1pr/ttyS000:07:431
12682992720,5cyclictest18232-21latency_hist23:47:370
12682992720,5cyclictest12681-21cyclictest22:44:510
100492270,3sleep20-21swapper/221:57:492
100492270,3sleep20-21swapper/221:57:492
257882260,3sleep20-21swapper/222:37:562
257882260,3sleep20-21swapper/222:37:562
170252260,3chrt0-21swapper/220:47:542
170252260,3chrt0-21swapper/220:47:542
158642260,3sleep3431rcuc/323:37:553
158642260,3sleep3431rcuc/323:37:553
158642260,3sleep3431rcuc/323:37:553
158642260,3sleep3431rcuc/323:37:553
1269099268,14cyclictest59232chrt00:38:053
1269099268,14cyclictest59232chrt00:38:053
1269099268,14cyclictest59232chrt00:38:053
1269099267,15cyclictest234082chrt19:40:253
1269099267,15cyclictest234082chrt19:40:253
1269099267,14cyclictest263252sleep321:12:553
1269099267,14cyclictest263252sleep321:12:553
1269099265,17cyclictest315122chrt22:54:183
1269099265,17cyclictest315122chrt22:54:183
1269099261,3cyclictest172052sleep320:50:043
1269099261,3cyclictest172052sleep320:50:043
1269099261,11cyclictest22566-21chrt22:32:393
1269099261,11cyclictest22566-21chrt22:32:393
1268399268,6cyclictest0-21swapper/120:36:271
12683992622,3cyclictest12681-21cyclictest19:36:431
12683992621,3cyclictest14379-21cron23:37:321
12683992621,3cyclictest14379-21cron23:37:321
12683992620,3cyclictest0-21swapper/121:52:321
12683992620,3cyclictest0-21swapper/121:52:321
12683992619,5cyclictest23232-21cpu21:07:431
12683992619,5cyclictest23232-21cpu21:07:431
12683992618,2cyclictest181rcu_preempt20:47:451
12683992618,2cyclictest181rcu_preempt20:47:451
12683992617,2cyclictest181rcu_preempt00:07:531
12683992617,2cyclictest181rcu_preempt00:07:531
12683992616,7cyclictest5699-21latency_hist21:47:391
12683992616,7cyclictest5699-21latency_hist21:47:391
12683992616,7cyclictest5699-21latency_hist21:47:391
12683992612,7cyclictest29-21ksoftirqd/123:27:451
12683992612,7cyclictest29-21ksoftirqd/123:27:451
12683992612,3cyclictest90-1pr/ttyS000:22:531
12683992612,3cyclictest90-1pr/ttyS000:22:531
12683992612,3cyclictest90-1pr/ttyS000:22:531
12683992611,3cyclictest90-1pr/ttyS022:47:481
12683992611,3cyclictest90-1pr/ttyS022:47:481
12683992610,2cyclictest181rcu_preempt22:55:331
12683992610,2cyclictest181rcu_preempt22:55:331
1268299268,2cyclictest0-21swapper/020:20:280
1268299268,2cyclictest0-21swapper/020:20:280
1268299262,7cyclictest848-21/usr/sbin/munin00:27:510
1268299262,7cyclictest848-21/usr/sbin/munin00:27:510
12682992621,3cyclictest4234-21head21:42:440
12682992620,5cyclictest23672-21threads22:32:540
12682992620,5cyclictest23672-21threads22:32:540
12682992620,5cyclictest23672-21threads22:32:540
12682992620,4cyclictest17562-21unixbench_multi19:23:090
12682992620,4cyclictest17562-21unixbench_multi19:23:090
1269099258,13cyclictest0-21swapper/320:00:473
1269099258,13cyclictest0-21swapper/320:00:473
1269099251,7cyclictest4850-21chrt23:11:143
1269099251,7cyclictest4850-21chrt23:11:143
1269099251,7cyclictest4850-21chrt23:11:143
1269099251,4cyclictest0-21swapper/300:22:503
1269099251,4cyclictest0-21swapper/300:22:503
1269099251,4cyclictest0-21swapper/300:22:503
1269099251,3cyclictest92932sleep321:57:173
1269099251,3cyclictest92932sleep321:57:173
1269099251,3cyclictest92932sleep321:57:173
1269099251,3cyclictest105722sleep323:26:543
1269099251,3cyclictest105722sleep323:26:543
1269099251,18cyclictest2002-21chrt20:07:553
1269099251,18cyclictest2002-21chrt20:07:553
1269099251,18cyclictest2002-21chrt20:07:553
1268399258,3cyclictest181rcu_preempt00:37:451
1268399257,2cyclictest0-21swapper/119:42:481
1268399257,2cyclictest0-21swapper/119:42:481
1268399257,2cyclictest0-21swapper/119:42:481
1268399255,10cyclictest90-1pr/ttyS020:56:541
1268399255,10cyclictest90-1pr/ttyS020:56:541
12683992520,4cyclictest575-21thd22:42:581
12683992519,4cyclictest90-1pr/ttyS023:19:141
12683992519,4cyclictest90-1pr/ttyS023:19:141
12683992518,6cyclictest8555-21gltestperf20:27:451
12683992518,6cyclictest8555-21gltestperf20:27:451
12683992518,5cyclictest6680-21gltestperf20:22:431
12683992518,5cyclictest6680-21gltestperf20:22:431
12683992518,5cyclictest28054-21gltestperf00:12:461
12683992518,5cyclictest28054-21gltestperf00:12:461
12683992516,2cyclictest181rcu_preempt19:37:431
12683992516,2cyclictest181rcu_preempt19:37:431
12683992513,3cyclictest90-1pr/ttyS021:17:501
12683992513,3cyclictest90-1pr/ttyS021:17:501
12683992512,4cyclictest21181-21kworker/u9:2-xprtiod21:35:561
12683992512,4cyclictest21181-21kworker/u9:2-xprtiod21:35:561
1268399251,14cyclictest3353-21open_files20:12:501
1268399251,14cyclictest3353-21open_files20:12:501
12683992510,4cyclictest90-1pr/ttyS000:32:451
1268299256,4cyclictest5262-21df23:12:420
1268299256,4cyclictest5262-21df23:12:420
1268299256,4cyclictest5262-21df23:12:420
12682992519,3cyclictest0-21swapper/020:57:450
12682992519,3cyclictest0-21swapper/020:57:450
12682992513,6cyclictest90-1pr/ttyS021:38:030
12682992513,6cyclictest90-1pr/ttyS021:38:030
12682992512,10cyclictest17906-21fw_packets20:52:440
12682992512,10cyclictest17906-21fw_packets20:52:440
13352240,3sleep20-21swapper/200:27:472
13352240,3sleep20-21swapper/200:27:472
1269099247,14cyclictest44822chrt21:42:463
1269099247,14cyclictest44822chrt21:42:463
1269099247,14cyclictest44822chrt21:42:463
1269099247,13cyclictest116562chrt20:36:173
1269099241,9cyclictest30713-21chrt21:27:433
1269099241,9cyclictest30713-21chrt21:27:433
1269099241,9cyclictest24768-21chrt22:37:453
1269099241,9cyclictest24768-21chrt22:37:453
1269099241,8cyclictest31205-21chrt21:27:473
1269099241,8cyclictest31205-21chrt21:27:473
1269099241,8cyclictest19860-21chrt23:50:543
1269099241,8cyclictest19860-21chrt23:50:543
1269099241,8cyclictest17715-21chrt19:24:013
1269099241,8cyclictest17715-21chrt19:24:013
1269099241,8cyclictest12999-21chrt23:32:453
1269099241,8cyclictest12999-21chrt23:32:453
1269099241,7cyclictest9541-21chrt20:28:013
1269099241,7cyclictest9541-21chrt20:28:013
1269099241,7cyclictest18275-21chrt19:27:433
1269099241,7cyclictest18275-21chrt19:27:433
1269099241,7cyclictest18275-21chrt19:27:433
1269099241,4cyclictest295742sleep322:48:193
1269099241,4cyclictest295742sleep322:48:193
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional