You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-12 - 01:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #4, slot #1

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1s.osadl.org (updated Mon May 11, 2026 12:44:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
47612980,2chrt4755-21gzip12:37:400
47612980,2chrt4755-21gzip12:37:400
255292910,2chrt25496-21gltestperf07:47:360
57602830,2sleep30-21swapper/312:42:223
57602830,2sleep30-21swapper/312:42:223
265782780,3sleep10-21swapper/112:07:461
265782780,3sleep10-21swapper/112:07:461
265782780,3sleep10-21swapper/112:07:461
79002730,3sleep10-21swapper/109:52:431
187942690,3sleep112306-21cyclictest11:47:441
187942690,3sleep112306-21cyclictest11:47:441
266292680,3sleep125897-21/usr/sbin/munin10:42:401
266292680,3sleep125897-21/usr/sbin/munin10:42:401
199092630,3chrt0-21swapper/108:57:441
199092630,3chrt0-21swapper/108:57:441
97022580,2chrt0-21swapper/309:57:403
97022580,2chrt0-21swapper/309:57:403
227842470,3sleep20-21swapper/209:07:342
227842470,3sleep20-21swapper/209:07:342
94212450,2sleep00-21swapper/009:57:370
94212450,2sleep00-21swapper/009:57:370
271612420,3sleep30-21swapper/309:17:403
271612420,3sleep30-21swapper/309:17:403
177622410,2sleep30-21swapper/307:27:333
284152390,3sleep20-21swapper/209:22:332
284152390,3sleep20-21swapper/209:22:332
12336993930,7cyclictest32189-21netstat10:57:411
12336993930,7cyclictest32189-21netstat10:57:411
28272380,4sleep10-21swapper/108:12:421
140032370,2sleep014005-21grep10:07:460
140032370,2sleep014005-21grep10:07:460
140032370,2sleep014005-21grep10:07:460
128522370,3sleep20-21swapper/211:32:382
12336993729,6cyclictest749-21runrttasks10:22:221
12336993729,6cyclictest749-21runrttasks10:22:221
166432350,3sleep20-21swapper/211:42:392
166432350,3sleep20-21swapper/211:42:392
12336993527,6cyclictest21784-21gltestperf07:37:351
12336993527,6cyclictest21784-21gltestperf07:37:351
12336993427,5cyclictest1867-21idleruntime-cro08:12:221
12336993427,5cyclictest1867-21idleruntime-cro08:12:221
12334993428,4cyclictest12306-21cyclictest08:42:220
12334993428,4cyclictest12306-21cyclictest08:42:220
144552330,2sleep20-21swapper/211:37:362
12336993226,4cyclictest5626-21cat08:22:221
12336993226,4cyclictest5626-21cat08:22:221
12334993224,6cyclictest9023-21apache208:42:490
12334993224,6cyclictest9023-21apache208:42:490
81962310,3sleep10-21swapper/108:27:371
81962310,3sleep10-21swapper/108:27:371
234222310,3chrt0-21swapper/110:32:471
234222310,3chrt0-21swapper/110:32:471
198902310,2chrt0-21swapper/307:32:363
198902310,2chrt0-21swapper/307:32:363
12336993126,3cyclictest32090-21unixbench-2d12:22:461
12336993126,3cyclictest32090-21unixbench-2d12:22:461
12336993126,3cyclictest32090-21unixbench-2d12:22:461
12336993126,3cyclictest14129-21ntp_offset08:42:411
12336993126,3cyclictest14129-21ntp_offset08:42:411
12336993125,4cyclictest1346-21avahi-daemon09:22:431
12336993125,4cyclictest1346-21avahi-daemon09:22:431
12334993125,3cyclictest0-21swapper/007:57:430
99112300,3sleep20-21swapper/208:32:362
99112300,3sleep20-21swapper/208:32:362
12336993025,3cyclictest547-21rs:main1
12336993025,3cyclictest547-21rs:main1
12336993025,3cyclictest12306-21cyclictest09:30:391
12336993025,3cyclictest12306-21cyclictest09:30:391
12336993025,2cyclictest0-21swapper/110:12:221
12336993025,2cyclictest0-21swapper/110:12:221
12336993025,2cyclictest0-21swapper/110:12:221
12336993024,4cyclictest15597-21latency_hist07:22:221
12336993023,5cyclictest25309-21tr09:12:411
12336993023,5cyclictest25309-21tr09:12:411
12336993022,6cyclictest5788-21sendmail12:42:221
12336993022,6cyclictest5788-21sendmail12:42:221
12334993025,3cyclictest0-21swapper/007:42:220
12334993025,3cyclictest0-21swapper/007:42:220
12334993023,5cyclictest17276-21ntp_offset10:17:400
12334993023,5cyclictest17276-21ntp_offset10:17:400
12337992915,10cyclictest21825-21chrt10:31:032
12337992915,10cyclictest21825-21chrt10:31:032
12337992915,10cyclictest21825-21chrt10:31:032
12336992923,5cyclictest2788-21latency12:32:391
12336992922,5cyclictest20571-21cut10:27:361
12336992922,5cyclictest20571-21cut10:27:361
12336992922,5cyclictest20571-21cut10:27:361
12336992921,6cyclictest11981-21sh11:32:351
12336992920,7cyclictest27720-21/usr/sbin/munin10:47:381
12336992920,7cyclictest27720-21/usr/sbin/munin10:47:381
12334992924,4cyclictest12306-21cyclictest08:12:460
12334992923,3cyclictest0-21swapper/011:12:230
12334992923,3cyclictest0-21swapper/011:12:230
12334992922,5cyclictest9815-21fschecks_time08:32:360
12334992922,5cyclictest9815-21fschecks_time08:32:360
12334992922,5cyclictest21721-21fschecks_time11:57:360
12334992922,5cyclictest21721-21fschecks_time11:57:360
12334992922,5cyclictest21721-21fschecks_time11:57:360
12334992922,5cyclictest21721-21fschecks_time11:57:360
12337992815,9cyclictest10282-21chrt11:27:312
12337992815,9cyclictest10282-21chrt11:27:312
12336992823,3cyclictest20064-21head11:52:361
12336992823,3cyclictest20064-21head11:52:361
12336992823,3cyclictest20064-21head11:52:361
12336992823,3cyclictest20064-21head11:52:361
12336992823,3cyclictest0-21swapper/109:57:421
12336992823,3cyclictest0-21swapper/109:57:421
12336992820,6cyclictest29994-21head10:52:361
12334992822,4cyclictest8434-21ntp_kernel_err08:27:410
12334992822,4cyclictest8434-21ntp_kernel_err08:27:410
12334992820,6cyclictest6822-21fschecks_time11:17:370
12336992723,2cyclictest0-21swapper/109:12:231
12336992723,2cyclictest0-21swapper/109:12:231
12336992721,3cyclictest0-21swapper/109:47:421
12336992721,3cyclictest0-21swapper/109:47:421
12336992721,3cyclictest0-21swapper/109:47:421
12336992720,4cyclictest0-21swapper/109:02:411
12336992720,4cyclictest0-21swapper/109:02:411
12334992722,4cyclictest32416-21latency_hist08:07:220
12334992722,4cyclictest32416-21latency_hist08:07:220
12334992721,5cyclictest12306-21cyclictest09:22:400
12334992721,5cyclictest12306-21cyclictest09:22:400
12334992720,5cyclictest20577-21sshd09:01:580
12334992720,5cyclictest20577-21sshd09:01:580
12334992720,5cyclictest0-21swapper/007:22:220
12334992719,6cyclictest5849-21unixbench-2d11:12:460
12334992719,6cyclictest22289-21fschecks_count10:32:350
12334992719,6cyclictest22289-21fschecks_count10:32:350
12334992719,6cyclictest18603-21fschecks_count10:22:350
12334992719,6cyclictest18603-21fschecks_count10:22:350
12337992616,7cyclictest2947-21chrt09:39:462
12337992616,7cyclictest2947-21chrt09:39:462
12337992615,8cyclictest6767-21chrt09:50:232
12337992615,8cyclictest6767-21chrt09:50:232
12337992615,8cyclictest6767-21chrt09:50:232
12337992615,8cyclictest32068-21chrt08:02:542
12337992615,8cyclictest32068-21chrt08:02:542
12336992620,5cyclictest3310-21kernelversion11:07:391
12336992620,5cyclictest3310-21kernelversion11:07:391
12336992620,4cyclictest28769-21latency_hist07:57:261
12336992620,4cyclictest0-21swapper/108:42:221
12336992620,4cyclictest0-21swapper/108:42:221
12336992620,3cyclictest0-21swapper/111:37:451
12336992619,5cyclictest29530-21kernelversion12:17:381
12336992619,5cyclictest12846-21cpuspeed10:07:341
12336992618,6cyclictest7418-21grep11:17:451
12334992621,3cyclictest12306-21cyclictest10:52:390
12334992621,3cyclictest0-21swapper/011:32:410
12334992619,5cyclictest11450-21timerwakeupswit11:27:450
12334992619,5cyclictest11450-21timerwakeupswit11:27:450
12334992619,5cyclictest11450-21timerwakeupswit11:27:450
12334992618,6cyclictest23798-21gltestperf12:02:370
12334992618,6cyclictest23798-21gltestperf12:02:370
12334992617,7cyclictest0-21swapper/009:12:220
12334992617,7cyclictest0-21swapper/009:12:220
12334992616,8cyclictest3560-21cut09:42:340
12334992616,8cyclictest3560-21cut09:42:340
12334992616,8cyclictest3560-21cut09:42:340
310962250,4sleep20-21swapper/209:27:442
310962250,4sleep20-21swapper/209:27:442
12337992515,7cyclictest22249-21chrt09:04:122
12337992515,7cyclictest22249-21chrt09:04:122
12337992515,7cyclictest17306-21chrt07:25:112
12337992515,7cyclictest17306-21chrt07:25:112
12337992515,7cyclictest16612-21chrt08:48:482
12337992515,7cyclictest16612-21chrt08:48:482
12337992514,7cyclictest30565-21chrt12:20:542
12337992514,6cyclictest21072-21chrt11:53:532
12337992514,6cyclictest21072-21chrt11:53:532
12337992514,6cyclictest21072-21chrt11:53:532
12337992514,6cyclictest21072-21chrt11:53:532
12336992519,4cyclictest0-21swapper/110:42:221
12336992519,4cyclictest0-21swapper/110:42:221
12336992517,6cyclictest9466-21sort11:22:441
12336992517,6cyclictest9466-21sort11:22:441
12336992517,6cyclictest10919-21/usr/sbin/munin10:02:411
12336992517,6cyclictest10919-21/usr/sbin/munin10:02:411
12336992517,6cyclictest10898-21sort11:27:381
12336992517,6cyclictest10898-21sort11:27:381
12336992517,6cyclictest10898-21sort11:27:381
12334992519,4cyclictest0-21swapper/011:52:350
12334992518,5cyclictest27375-21fschecks_time12:12:360
12334992514,3cyclictest181rcu_preempt11:02:350
12334992514,3cyclictest181rcu_preempt11:02:350
12334992514,3cyclictest181rcu_preempt11:02:350
166942240,21sleep016729-21ntp_states11:42:420
166942240,21sleep016729-21ntp_states11:42:420
12337992416,5cyclictest5654-21chrt12:41:042
12337992416,5cyclictest5654-21chrt12:41:042
12337992415,6cyclictest3501-21chrt08:14:352
12337992415,6cyclictest30315-21chrt07:59:362
12337992415,6cyclictest22940-21chrt07:40:512
12337992415,6cyclictest22940-21chrt07:40:512
12337992415,6cyclictest18549-21chrt08:54:472
12337992415,6cyclictest12947-21chrt08:40:122
12337992415,6cyclictest12947-21chrt08:40:122
12337992415,5cyclictest15250-21chrt07:18:032
12337992414,7cyclictest8535-21chrt09:54:092
12337992414,7cyclictest27509-21chrt10:47:032
12337992414,7cyclictest27509-21chrt10:47:032
12337992414,7cyclictest23580-21chrt10:34:412
12337992414,7cyclictest23580-21chrt10:34:412
12337992414,7cyclictest20953-21chrt07:34:112
12337992414,7cyclictest20953-21chrt07:34:112
12337992414,7cyclictest14143-21chrt10:08:242
12337992414,7cyclictest14143-21chrt10:08:242
12337992414,7cyclictest14143-21chrt10:08:242
12337992414,7cyclictest1165-21chrt09:35:562
12337992414,7cyclictest1165-21chrt09:35:562
12337992414,6cyclictest6140-21chrt11:14:532
12337992414,6cyclictest26152-21chrt09:16:222
12337992414,6cyclictest26152-21chrt09:16:222
12337992414,6cyclictest12447-21chrt10:05:552
12337992414,6cyclictest12447-21chrt10:05:552
12337992414,6cyclictest12447-21chrt10:05:552
12337992413,8cyclictest404-21chrt10:59:052
12337992413,8cyclictest404-21chrt10:59:052
12337992413,7cyclictest4310-21chrt11:10:392
12337992413,7cyclictest4310-21chrt11:10:392
12336992420,2cyclictest0-21swapper/108:32:431
12336992420,2cyclictest0-21swapper/108:32:431
12336992420,2cyclictest0-21swapper/108:27:221
12336992420,2cyclictest0-21swapper/108:27:221
12336992419,3cyclictest0-21swapper/107:26:581
12336992419,3cyclictest0-21swapper/107:26:581
12336992418,4cyclictest0-21swapper/111:02:361
12336992418,4cyclictest0-21swapper/111:02:361
12336992418,4cyclictest0-21swapper/111:02:361
12336992418,3cyclictest0-21swapper/108:02:461
12336992418,3cyclictest0-21swapper/108:02:461
12336992416,6cyclictest15397-21ntp_kernel_pll_10:12:421
12336992416,6cyclictest15397-21ntp_kernel_pll_10:12:421
12336992415,7cyclictest23551-21ls12:02:341
12336992415,7cyclictest23551-21ls12:02:341
12334992419,3cyclictest0-21swapper/009:52:450
12334992419,3cyclictest0-21swapper/009:37:440
12334992419,3cyclictest0-21swapper/009:37:440
12334992417,6cyclictest32206-21ntp_kernel_pll_10:57:420
12334992417,6cyclictest32206-21ntp_kernel_pll_10:57:420
12334992417,5cyclictest21935-21latency_hist10:32:220
12334992417,5cyclictest21935-21latency_hist10:32:220
12334992417,5cyclictest21935-21latency_hist10:32:220
12334992415,7cyclictest882-21needreboot08:07:400
12334992415,7cyclictest882-21needreboot08:07:400
12334992415,7cyclictest4194-21fschecks_time08:17:330
12334992415,7cyclictest4194-21fschecks_time08:17:330
12334992412,10cyclictest14678-21systemd-journal10:52:220
12334992412,10cyclictest14678-21systemd-journal10:52:220
100662240,3chrt0-21swapper/308:32:373
100662240,3chrt0-21swapper/308:32:373
12337992315,6cyclictest17887-21chrt10:18:412
12337992315,6cyclictest17887-21chrt10:18:412
12337992315,4cyclictest20567-21chrt09:01:512
12337992315,4cyclictest20567-21chrt09:01:512
12337992314,7cyclictest4752-21chrt09:43:442
12337992314,7cyclictest4752-21chrt09:43:442
12337992314,7cyclictest4752-21chrt09:43:442
12337992314,6cyclictest27809-21chrt09:18:432
12337992314,6cyclictest27809-21chrt09:18:432
12337992314,6cyclictest23353-21chrt12:02:292
12337992314,6cyclictest23353-21chrt12:02:292
12337992314,6cyclictest23353-21chrt12:02:292
12337992314,6cyclictest2244-21chrt11:02:592
12337992314,6cyclictest2244-21chrt11:02:592
12337992314,6cyclictest2244-21chrt11:02:592
12337992314,6cyclictest19860-21chrt10:24:442
12337992314,6cyclictest19860-21chrt10:24:442
12337992314,6cyclictest19223-21chrt11:49:022
12337992314,6cyclictest19223-21chrt11:49:022
12337992314,6cyclictest1686-21chrt12:28:182
12337992314,6cyclictest1686-21chrt12:28:182
12337992314,6cyclictest10377-21chrt09:58:352
12337992314,6cyclictest10377-21chrt09:58:352
12337992313,7cyclictest3705-21taskset12:35:172
12337992313,6cyclictest5419-21chrt08:19:542
12337992313,6cyclictest5419-21chrt08:19:542
12337992313,6cyclictest1618-21chrt08:09:252
12337992313,6cyclictest1618-21chrt08:09:252
12337992312,7cyclictest31035-21chrt10:54:272
12337992312,7cyclictest28430-21chrt07:54:172
12337992312,7cyclictest25603-21chrt10:41:492
12337992312,7cyclictest25603-21chrt10:41:492
12336992319,2cyclictest0-21swapper/107:57:451
12336992318,3cyclictest0-21swapper/111:42:411
12336992318,3cyclictest0-21swapper/111:42:411
12336992318,3cyclictest0-21swapper/109:42:351
12336992318,3cyclictest0-21swapper/109:42:351
12336992318,3cyclictest0-21swapper/109:42:351
12336992317,4cyclictest19778-21fschecks_time07:32:351
12336992317,4cyclictest19778-21fschecks_time07:32:351
12336992317,4cyclictest0-21swapper/107:32:221
12336992317,4cyclictest0-21swapper/107:32:221
12336992315,6cyclictest876-21cut12:27:381
12336992315,6cyclictest876-21cut12:27:381
12336992315,6cyclictest32035-21/usr/sbin/munin09:32:401
12336992315,6cyclictest32035-21/usr/sbin/munin09:32:401
12334992318,4cyclictest749-21runrttasks10:02:360
12334992318,4cyclictest749-21runrttasks10:02:360
12334992318,4cyclictest749-21runrttasks10:02:360
12334992318,4cyclictest12306-21cyclictest07:28:580
12334992318,4cyclictest12306-21cyclictest07:28:580
12334992318,3cyclictest17485-21sh07:27:220
12334992318,3cyclictest17485-21sh07:27:220
12334992318,3cyclictest0-21swapper/012:07:460
12334992318,3cyclictest0-21swapper/012:07:460
12334992318,3cyclictest0-21swapper/012:07:460
12334992318,3cyclictest0-21swapper/010:37:450
12334992318,3cyclictest0-21swapper/010:37:450
12334992318,3cyclictest0-21swapper/008:52:360
12334992317,4cyclictest7508-21wc08:27:220
12334992317,4cyclictest7508-21wc08:27:220
12334992317,4cyclictest12306-21cyclictest09:17:430
12334992317,4cyclictest12306-21cyclictest09:17:430
12334992316,5cyclictest14457-21gltestperf11:37:360
12334992315,6cyclictest22730-21unixbench-2d11:57:460
12334992315,6cyclictest22730-21unixbench-2d11:57:460
12334992315,6cyclictest22730-21unixbench-2d11:57:460
12337992217,3cyclictest264782sleep207:48:062
12337992213,7cyclictest28594-21chrt12:14:352
12337992213,6cyclictest29187-21chrt10:49:412
12337992213,6cyclictest29187-21chrt10:49:412
12337992213,6cyclictest24766-21chrt12:03:132
12337992213,6cyclictest24766-21chrt12:03:132
12337992213,6cyclictest13427-21chrt08:42:332
12337992213,6cyclictest13427-21chrt08:42:332
12337992213,6cyclictest11750-21chrt11:29:282
12337992213,6cyclictest11750-21chrt11:29:282
12337992213,6cyclictest11750-21chrt11:29:282
12337992211,7cyclictest7044-21chrt11:17:392
12336992218,2cyclictest0-21swapper/112:12:371
12336992218,2cyclictest0-21swapper/107:47:431
12336992217,3cyclictest0-21swapper/109:17:451
12336992217,3cyclictest0-21swapper/109:17:451
12336992217,2cyclictest0-21swapper/111:12:361
12336992216,3cyclictest0-21swapper/111:57:471
12336992216,3cyclictest0-21swapper/111:57:471
12336992216,3cyclictest0-21swapper/111:57:471
12336992214,6cyclictest19158-21ntp_kernel_pll_10:22:411
12336992214,6cyclictest19158-21ntp_kernel_pll_10:22:411
12334992218,2cyclictest0-21swapper/009:17:220
12334992218,2cyclictest0-21swapper/009:17:220
12334992217,3cyclictest12306-21cyclictest10:42:340
12334992217,3cyclictest12306-21cyclictest10:42:340
12334992217,3cyclictest0-21swapper/010:12:450
12334992217,3cyclictest0-21swapper/010:12:450
12334992217,3cyclictest0-21swapper/009:32:440
12334992217,3cyclictest0-21swapper/009:32:440
12334992216,5cyclictest31858-21latency_hist09:32:220
12334992216,5cyclictest31858-21latency_hist09:32:220
12334992216,5cyclictest31609-21needreboot12:22:410
12334992216,5cyclictest31609-21needreboot12:22:410
12334992216,5cyclictest31609-21needreboot12:22:410
12334992216,5cyclictest30494-21sshd12:19:580
12334992216,4cyclictest21202-21grep09:02:350
12334992216,4cyclictest21202-21grep09:02:350
12334992215,5cyclictest6926-21latency_hist09:52:350
12334992214,6cyclictest749-21runrttasks08:47:350
12334992214,6cyclictest749-21runrttasks08:47:350
12334992212,8cyclictest14678-21systemd-journal11:52:220
12337992115,3cyclictest32538-21chrt12:27:012
12337992115,3cyclictest32538-21chrt12:27:012
12337992115,3cyclictest32538-21chrt12:27:012
12337992112,6cyclictest9336-21chrt08:31:482
12337992112,6cyclictest9336-21chrt08:31:482
12337992112,6cyclictest24710-21chrt07:44:202
12337992112,6cyclictest24710-21chrt07:44:202
12337992112,6cyclictest16250-21taskset10:16:362
12337992112,6cyclictest16250-21taskset10:16:362
12337992111,6cyclictest7266-21taskset08:24:242
12337992111,6cyclictest7266-21taskset08:24:242
1233699212,4cyclictest17445-21sed08:52:341
1233499219,10cyclictest538-21cut12:27:350
1233499219,10cyclictest538-21cut12:27:350
12334992117,3cyclictest19928-21head07:32:360
12334992117,3cyclictest19928-21head07:32:360
12334992117,2cyclictest0-21swapper/011:22:450
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional