You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-16 - 15:01

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #4, slot #1

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1s.osadl.org (updated Thu Jul 16, 2026 12:44:19)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2320221090,4sleep20-21swapper/211:43:362
242772910,2sleep20-21swapper/210:18:422
242772910,2sleep20-21swapper/210:18:422
242772910,2sleep20-21swapper/210:18:422
83602880,2sleep30-21swapper/308:08:463
83602880,2sleep30-21swapper/308:08:463
212252860,3sleep30-21swapper/308:43:433
306242850,2sleep20-21swapper/209:08:432
306242850,2sleep20-21swapper/209:08:432
306242850,2sleep20-21swapper/209:08:432
16422820,3sleep30-21swapper/312:13:323
16422820,3sleep30-21swapper/312:13:323
29342810,3chrt0-21swapper/312:13:483
29342810,3chrt0-21swapper/312:13:483
29342810,3chrt0-21swapper/312:13:483
7802720,2sleep00-21swapper/010:43:360
23552710,2sleep02357-21tail12:13:410
23552710,2sleep02357-21tail12:13:410
23552710,2sleep02357-21tail12:13:410
311122650,2sleep131138-21ntp_states12:03:421
311122650,2sleep131138-21ntp_states12:03:421
269352640,6sleep026939-21gltestperf11:53:350
41912630,1sleep016-21ksoftirqd/010:53:330
41912630,1sleep016-21ksoftirqd/010:53:330
251452570,4chrt25144-21basename10:23:220
251452570,4chrt25144-21basename10:23:220
251452570,4chrt25144-21basename10:23:220
263782550,3chrt0-21swapper/007:33:260
263782550,3chrt0-21swapper/007:33:260
19277994620,24cyclictest28085-21sshd11:55:501
19277994620,24cyclictest28085-21sshd11:55:501
143562460,3sleep30-21swapper/309:53:343
143562460,3sleep30-21swapper/309:53:343
143562460,3sleep30-21swapper/309:53:343
143562460,3sleep30-21swapper/309:53:343
19277994522,21cyclictest23630-21sshd08:50:501
19277994522,21cyclictest23630-21sshd08:50:501
19277994319,5cyclictest1269-21sshd07:52:501
19277994319,5cyclictest1269-21sshd07:52:501
103912430,2sleep20-21swapper/212:33:452
103912430,2sleep20-21swapper/212:33:452
103912430,2sleep20-21swapper/212:33:452
273022420,3sleep30-21swapper/311:53:383
273022420,3sleep30-21swapper/311:53:383
219702420,2sleep021973-21sed10:13:370
219702420,2sleep021973-21sed10:13:370
45292410,2sleep04693-21sed12:18:440
45292410,2sleep04693-21sed12:18:440
45292410,2sleep04693-21sed12:18:440
135952400,3sleep00-21swapper/011:18:330
135952400,3sleep00-21swapper/011:18:330
19277993919,3cyclictest23257-21latency_hist10:18:251
19277993919,3cyclictest23257-21latency_hist10:18:251
126632390,3sleep20-21swapper/211:13:452
126632390,3sleep20-21swapper/211:13:452
184542360,2sleep20-21swapper/210:03:382
184542360,2sleep20-21swapper/210:03:382
19277993427,5cyclictest1983-21threads09:18:451
19277993427,5cyclictest1983-21threads09:18:451
19277993427,5cyclictest1983-21threads09:18:451
19277993427,5cyclictest19269-21cyclictest10:18:431
19277993427,5cyclictest19269-21cyclictest10:18:431
19277993427,5cyclictest19269-21cyclictest10:18:431
19271993428,4cyclictest31499-21timerwakeupswit12:03:450
19271993428,4cyclictest31499-21timerwakeupswit12:03:450
19271993427,5cyclictest21378-21gltestperf11:38:350
19271993427,5cyclictest21378-21gltestperf11:38:350
19277993327,4cyclictest19269-21cyclictest11:38:451
19277993327,4cyclictest19269-21cyclictest11:38:451
19277993327,4cyclictest19269-21cyclictest11:38:451
19277993327,4cyclictest19269-21cyclictest11:38:451
19271993326,5cyclictest24916-21fschecks_count11:48:340
19271993326,5cyclictest24916-21fschecks_count11:48:340
19271993223,7cyclictest18377-21kernelversion10:03:380
19271993223,7cyclictest18377-21kernelversion10:03:380
19277993126,3cyclictest7704-21sendmail11:03:231
19277993122,7cyclictest3618-21fschecks_time07:58:371
19277993122,7cyclictest3618-21fschecks_time07:58:371
19277993122,7cyclictest3618-21fschecks_time07:58:371
19277993122,7cyclictest23050-21fschecks_time07:23:371
19271993123,6cyclictest1466-21idleruntime-cro12:13:220
19271993123,6cyclictest1466-21idleruntime-cro12:13:220
19271993122,7cyclictest10290-21kernelversion11:08:360
19271993122,7cyclictest10290-21kernelversion11:08:360
142182310,3sleep20-21swapper/211:18:402
142182310,3sleep20-21swapper/211:18:402
142182310,3sleep20-21swapper/211:18:402
22362300,2sleep12255-21cat10:48:331
22362300,2sleep12255-21cat10:48:331
22362300,2sleep12255-21cat10:48:331
19277993025,4cyclictest30960-21sshd09:09:021
19277993025,4cyclictest30960-21sshd09:09:021
19277993025,4cyclictest30960-21sshd09:09:021
19277993024,5cyclictest9179-21needreboot09:38:411
19277993024,5cyclictest9179-21needreboot09:38:411
19277993024,5cyclictest9179-21needreboot09:38:411
19277993023,5cyclictest6030-21latency12:23:361
19277993023,5cyclictest6030-21latency12:23:361
19277993023,5cyclictest19798-21missed_timers11:33:391
19277993023,5cyclictest19798-21missed_timers11:33:391
19277993023,5cyclictest19798-21missed_timers11:33:391
19277993023,5cyclictest17587-21threads08:33:441
19277993023,5cyclictest17587-21threads08:33:441
19277993021,7cyclictest494-21netstat12:08:411
19277993021,7cyclictest494-21netstat12:08:411
19271993025,3cyclictest18804-21head08:38:360
19271993025,3cyclictest18804-21head08:38:360
19271993024,4cyclictest7507-21gltestperf08:08:370
19271993024,4cyclictest7507-21gltestperf08:08:370
19271993024,4cyclictest14678-21systemd-journal12:38:220
19271993024,4cyclictest14678-21systemd-journal12:38:220
19271993024,4cyclictest14678-21systemd-journal12:38:220
19271993022,6cyclictest4794-21needreboot10:53:400
19271993022,6cyclictest4794-21needreboot10:53:400
19277992924,3cyclictest24785-21ntp_offset08:53:401
19277992924,3cyclictest11933-21kworker/u9:3-xprtiod09:58:221
19277992924,3cyclictest11933-21kworker/u9:3-xprtiod09:58:221
19277992922,5cyclictest14678-21systemd-journal10:07:501
19277992922,5cyclictest14678-21systemd-journal10:07:501
19271992924,3cyclictest8197-21gltestperf11:03:350
19271992924,3cyclictest8197-21gltestperf11:03:350
19271992923,5cyclictest31799-21latency_hist09:13:340
19271992923,5cyclictest31799-21latency_hist09:13:340
19271992923,5cyclictest31799-21latency_hist09:13:340
19271992923,4cyclictest17085-21idleruntime-cro11:28:220
19271992923,4cyclictest17085-21idleruntime-cro11:28:220
19271992923,4cyclictest17085-21idleruntime-cro11:28:220
19271992922,5cyclictest547-21rs:main0
19271992922,5cyclictest547-21rs:main0
19271992922,5cyclictest29387-21sendmail_mailst11:58:420
1927799287,3cyclictest20402grep12:13:381
1927799287,3cyclictest20402grep12:13:381
1927799287,3cyclictest20402grep12:13:381
19277992824,3cyclictest17643-21sendmail10:03:221
19277992822,4cyclictest1346-21avahi-daemon07:38:431
19277992822,4cyclictest1346-21avahi-daemon07:38:431
19277992821,6cyclictest14452-21gltestperf09:53:351
19277992821,6cyclictest14452-21gltestperf09:53:351
19277992821,6cyclictest14452-21gltestperf09:53:351
19277992821,6cyclictest14452-21gltestperf09:53:351
19277992821,5cyclictest15075-21sshd11:21:501
19277992821,5cyclictest15075-21sshd11:21:501
19277992821,5cyclictest15075-21sshd11:21:501
19277992820,6cyclictest3074-21proc_pri10:48:431
19277992820,6cyclictest3074-21proc_pri10:48:431
19277992820,6cyclictest12367-21netstat11:13:411
19277992820,6cyclictest12367-21netstat11:13:411
19277992814,3cyclictest181rcu_preempt12:33:371
19277992814,3cyclictest181rcu_preempt12:33:371
19277992814,3cyclictest181rcu_preempt12:33:371
1927199287,3cyclictest226382cat08:48:370
1927199287,3cyclictest226382cat08:48:370
19271992823,2cyclictest0-21swapper/008:53:420
19271992822,4cyclictest9761-21netstat08:13:400
19271992822,4cyclictest9761-21netstat08:13:400
19271992822,4cyclictest17728-21latency_hist10:03:250
19271992822,3cyclictest0-21swapper/007:48:330
19271992822,3cyclictest0-21swapper/007:48:330
19271992821,5cyclictest12901-21latency_hist12:43:320
19271992821,5cyclictest12901-21latency_hist12:43:320
19271992819,7cyclictest6692-21unixbench_singl12:23:440
19271992819,7cyclictest6692-21unixbench_singl12:23:440
19271992818,7cyclictest9498-21gltestperf12:33:350
19271992816,10cyclictest0-21swapper/009:43:220
19271992816,10cyclictest0-21swapper/009:43:220
19271992816,10cyclictest0-21swapper/009:43:220
48172270,4sleep30-21swapper/309:28:343
48172270,4sleep30-21swapper/309:28:343
48172270,4sleep30-21swapper/309:28:343
19277992723,3cyclictest4292-21ntp_offset12:18:401
19277992723,3cyclictest4292-21ntp_offset12:18:401
19277992723,3cyclictest4292-21ntp_offset12:18:401
19277992722,3cyclictest8973-21cron12:33:221
19277992722,3cyclictest8973-21cron12:33:221
19277992722,3cyclictest8973-21cron12:33:221
19277992720,5cyclictest3390-21latency09:23:381
19277992720,5cyclictest3390-21latency09:23:381
19277992720,5cyclictest3390-21latency09:23:381
19277992720,5cyclictest3390-21latency09:23:381
19277992719,6cyclictest30934-21/usr/sbin/munin10:38:421
19277992719,6cyclictest30934-21/usr/sbin/munin10:38:421
19277992717,8cyclictest1125-21fschecks_count09:18:351
19277992717,8cyclictest1125-21fschecks_count09:18:351
19271992721,4cyclictest12881-21needreboot09:48:390
19271992721,3cyclictest0-21swapper/007:23:450
19271992720,5cyclictest550-21cron08:28:330
19271992720,5cyclictest550-21cron08:28:330
19271992719,6cyclictest31634-21needreboot10:38:410
19271992719,5cyclictest19269-21cyclictest09:20:500
19271992719,5cyclictest19269-21cyclictest09:20:500
19271992719,5cyclictest19269-21cyclictest09:20:500
19271992713,11cyclictest4002-21memory07:58:400
19271992713,11cyclictest4002-21memory07:58:400
19271992713,11cyclictest4002-21memory07:58:400
19277992621,4cyclictest19269-21cyclictest09:07:361
19277992621,4cyclictest19269-21cyclictest09:07:361
19277992621,3cyclictest11601-21kworker/u9:2-rpciod08:43:361
19277992620,4cyclictest1346-21avahi-daemon11:08:421
19277992620,4cyclictest1346-21avahi-daemon11:08:421
19277992619,5cyclictest26589-21needreboot08:58:401
19277992619,5cyclictest26589-21needreboot08:58:401
19277992618,6cyclictest8236-21head11:03:351
19277992618,6cyclictest8236-21head11:03:351
19277992618,6cyclictest27487-21fschecks_time10:28:361
19277992614,10cyclictest30535-21sshd10:35:501
19277992614,10cyclictest30535-21sshd10:35:501
19277992614,10cyclictest30535-21sshd10:35:501
19271992622,2cyclictest0-21swapper/008:33:410
19271992622,2cyclictest0-21swapper/008:33:410
19271992621,4cyclictest28866-21kernelversion07:38:370
19271992621,4cyclictest28866-21kernelversion07:38:370
19271992621,3cyclictest0-21swapper/011:43:330
19271992621,3cyclictest0-21swapper/011:43:330
19271992621,3cyclictest0-21swapper/011:43:330
19271992621,3cyclictest0-21swapper/011:43:330
19271992620,5cyclictest31299-21sh09:13:370
19271992620,5cyclictest31299-21sh09:13:370
19271992620,5cyclictest31299-21sh09:13:370
19271992620,5cyclictest31299-21sh09:13:370
19271992618,6cyclictest6959-21cut10:58:430
19277992521,3cyclictest16184-21ntp_offset11:23:411
19277992521,3cyclictest16184-21ntp_offset11:23:411
19277992521,3cyclictest16184-21ntp_offset11:23:411
19277992521,2cyclictest0-21swapper/108:43:221
19277992521,2cyclictest0-21swapper/108:43:221
19277992518,5cyclictest15582-21sendmail_mailqu08:28:421
19277992518,5cyclictest15582-21sendmail_mailqu08:28:421
19271992521,3cyclictest19269-21cyclictest11:32:010
19271992521,3cyclictest19269-21cyclictest11:32:010
19271992520,4cyclictest30743-21kernelversion07:43:370
19271992520,3cyclictest0-21swapper/011:53:440
19271992520,3cyclictest0-21swapper/011:53:440
19271992519,5cyclictest865-21unixbench_multi07:48:450
19271992519,5cyclictest865-21unixbench_multi07:48:450
19271992519,3cyclictest0-21swapper/011:18:380
19271992519,3cyclictest0-21swapper/011:18:380
19271992519,3cyclictest0-21swapper/011:18:380
19271992519,3cyclictest0-21swapper/008:43:380
1927799241,7cyclictest249532tail07:28:351
1927799241,7cyclictest249532tail07:28:351
19277992417,5cyclictest25870-21kernelversion10:23:371
19277992417,5cyclictest2356-21processes07:53:411
19277992417,5cyclictest2356-21processes07:53:411
19277992417,5cyclictest18316-21threads11:28:441
19277992417,5cyclictest18316-21threads11:28:441
19277992417,5cyclictest11633-21unixbench_singl09:43:471
19277992417,5cyclictest11633-21unixbench_singl09:43:471
19277992417,5cyclictest11633-21unixbench_singl09:43:471
19277992416,6cyclictest13710-21grep08:23:441
19277992416,6cyclictest13710-21grep08:23:441
19277992415,7cyclictest11813-21tr12:38:411
19277992415,7cyclictest11813-21tr12:38:411
19271992420,2cyclictest0-21swapper/010:43:430
19271992420,2cyclictest0-21swapper/010:43:430
19271992420,2cyclictest0-21swapper/010:43:430
19271992420,2cyclictest0-21swapper/008:23:460
19271992420,2cyclictest0-21swapper/008:23:460
19271992419,3cyclictest0-21swapper/012:28:410
19271992419,3cyclictest0-21swapper/012:28:410
19271992418,5cyclictest13409-21unixbench-2d09:48:450
19271992418,5cyclictest13409-21unixbench-2d09:48:450
19271992418,5cyclictest13409-21unixbench-2d09:48:450
19271992418,5cyclictest13409-21unixbench-2d09:48:450
19271992418,4cyclictest0-21swapper/009:23:370
19271992418,4cyclictest0-21swapper/009:23:370
19271992418,4cyclictest0-21swapper/009:23:370
19271992418,4cyclictest0-21swapper/009:23:370
19271992418,3cyclictest0-21swapper/010:28:360
19271992418,3cyclictest0-21swapper/009:48:350
19271992418,3cyclictest0-21swapper/009:48:350
19271992417,5cyclictest6030-21ntp_kernel_err08:03:410
19271992417,5cyclictest6030-21ntp_kernel_err08:03:410
83722230,2sleep20-21swapper/211:03:362
83722230,2sleep20-21swapper/211:03:362
19277992317,5cyclictest28869-21cron10:33:221
19277992317,5cyclictest28869-21cron10:33:221
19277992317,3cyclictest0-21swapper/110:08:401
19277992317,3cyclictest0-21swapper/110:08:401
19277992316,5cyclictest550-21cron08:08:221
19277992316,5cyclictest550-21cron08:08:221
19277992315,6cyclictest29596-21sort11:58:441
19277992311,10cyclictest32119-21/usr/sbin/munin09:13:371
19277992311,10cyclictest32119-21/usr/sbin/munin09:13:371
19271992318,4cyclictest19269-21cyclictest10:33:390
19271992318,4cyclictest19269-21cyclictest10:33:390
19271992318,4cyclictest19269-21cyclictest10:33:390
19271992318,3cyclictest0-21swapper/011:33:450
19271992318,3cyclictest0-21swapper/010:28:470
19271992318,3cyclictest0-21swapper/010:28:470
19271992318,3cyclictest0-21swapper/009:38:360
19271992318,3cyclictest0-21swapper/009:38:360
19271992318,3cyclictest0-21swapper/009:38:360
19271992318,3cyclictest0-21swapper/008:58:330
19271992318,3cyclictest0-21swapper/008:58:330
19271992317,4cyclictest0-21swapper/009:53:430
19271992317,4cyclictest0-21swapper/009:53:430
19277992216,3cyclictest0-21swapper/110:53:451
19277992216,3cyclictest0-21swapper/110:53:451
19277992216,3cyclictest0-21swapper/109:38:231
19277992216,3cyclictest0-21swapper/109:38:231
19277992216,3cyclictest0-21swapper/109:38:231
19271992218,2cyclictest0-21swapper/007:33:430
19271992218,2cyclictest0-21swapper/007:33:430
19271992217,3cyclictest19269-21cyclictest08:18:360
19271992217,3cyclictest19269-21cyclictest08:18:360
19271992213,7cyclictest5379-21ls09:28:380
19271992213,7cyclictest5379-21ls09:28:380
19271992212,8cyclictest0-21swapper/010:28:220
19277992117,2cyclictest0-21swapper/111:47:451
19277992117,2cyclictest0-21swapper/111:47:451
19277992117,2cyclictest0-21swapper/108:08:411
19277992117,2cyclictest0-21swapper/108:08:411
19277992115,3cyclictest0-21swapper/111:53:321
19277992115,3cyclictest0-21swapper/111:53:321
19277992115,3cyclictest0-21swapper/111:53:321
19277992114,5cyclictest11182-21fschecks_time08:18:361
19277992114,5cyclictest11182-21fschecks_time08:18:361
19277992111,8cyclictest14678-21systemd-journal09:32:501
19277992111,8cyclictest14678-21systemd-journal09:32:501
19271992116,3cyclictest0-21swapper/009:03:430
19271992116,3cyclictest0-21swapper/009:03:430
19271992112,7cyclictest14678-21systemd-journal11:48:360
19271992112,7cyclictest14678-21systemd-journal11:48:360
19281992014,3cyclictest250422chrt11:48:342
19281992014,3cyclictest250422chrt11:48:342
19277992016,2cyclictest0-21swapper/107:43:341
19277992016,2cyclictest0-21swapper/107:43:341
19277992016,2cyclictest0-21swapper/107:43:341
19277992014,4cyclictest9323-21fschecks_time08:13:351
19277992014,4cyclictest9323-21fschecks_time08:13:351
19277992013,5cyclictest0-21swapper/107:33:441
19277992013,5cyclictest0-21swapper/107:33:441
1927199209,9cyclictest2294-21ntpq07:53:400
1927199209,9cyclictest2294-21ntpq07:53:400
50392180,2sleep20-21swapper/210:53:432
50392180,2sleep20-21swapper/210:53:432
36492180,3sleep20-21swapper/212:18:332
36492180,3sleep20-21swapper/212:18:332
1928899183,9cyclictest0-21swapper/309:03:503
1928899183,9cyclictest0-21swapper/309:03:503
19288991813,3cyclictest6982sleep307:48:443
19288991813,3cyclictest6982sleep307:48:443
1928899181,13cyclictest21941-21chrt11:38:423
1928899181,13cyclictest21941-21chrt11:38:423
1928899181,13cyclictest21941-21chrt11:38:423
1928899181,13cyclictest21941-21chrt11:38:423
1928899181,12cyclictest5580-21chrt12:23:333
1928899181,12cyclictest5580-21chrt12:23:333
1928899181,12cyclictest5580-21chrt12:23:333
19281991812,3cyclictest235732chrt10:18:342
19281991812,3cyclictest235732chrt10:18:342
19281991811,5cyclictest273232sleep210:28:362
1928899173,9cyclictest0-21swapper/309:29:053
1928899173,9cyclictest0-21swapper/309:29:053
1928899171,11cyclictest18056-21chrt11:28:413
1928899171,11cyclictest18056-21chrt11:28:413
1928899171,11cyclictest16751-21chrt08:33:343
1928899171,11cyclictest16751-21chrt08:33:343
1928899171,10cyclictest8017-21chrt11:03:333
1928899171,10cyclictest8017-21chrt11:03:333
19288991710,4cyclictest19912chrt07:53:353
19288991710,4cyclictest19912chrt07:53:353
1928199171,14cyclictest16708-21chrt11:23:462
1928199171,14cyclictest16708-21chrt11:23:462
1928199171,14cyclictest16708-21chrt11:23:462
1928899162,9cyclictest111452sleep311:09:373
1928899162,9cyclictest111452sleep311:09:373
1928899161,11cyclictest4311-21chrt07:58:453
1928899161,11cyclictest4311-21chrt07:58:453
1928899161,11cyclictest4311-21chrt07:58:453
1928899161,11cyclictest31834-21chrt09:13:343
1928899161,11cyclictest31834-21chrt09:13:343
1928899161,11cyclictest31834-21chrt09:13:343
1928899161,11cyclictest30167-21chrt12:03:233
1928899161,11cyclictest23394-21chrt10:18:313
1928899161,11cyclictest23394-21chrt10:18:313
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional