You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-23 - 20:38

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Thu Jul 23, 2026 12:43:12)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
671268995534,14cyclictest759798-21sshd10:35:481
671268995042,5cyclictest806416-21ssh11:55:011
671267995034,13cyclictest761585-21ssh10:40:000
671267994841,5cyclictest824343-21munin-plugin-st12:25:000
671268994537,5cyclictest1002-21mta-sts-daemon12:20:521
671268994536,6cyclictest4066570-21cups-browsed09:55:521
671268994533,4cyclictest1000-21mta-sts-daemon11:36:451
671267994527,14cyclictest695238-40mandb08:28:460
671267994523,12cyclictest151ktimers/007:30:460
671269994436,5cyclictest769070-21threads10:50:562
671268994438,3cyclictest0-21swapper/109:45:531
671268994438,3cyclictest0-21swapper/109:45:531
671268994436,5cyclictest0-21swapper/112:05:471
671268994337,3cyclictest0-21swapper/110:20:431
671268994335,5cyclictest803433-21systemd-run11:50:011
671268994335,5cyclictest768438-21kernelversion10:50:411
671268994333,7cyclictest421-21dbus-daemon09:20:011
671269994232,7cyclictest420-21avahi-daemon12:33:322
671268994235,4cyclictest691262-21fschecks_count08:15:381
671268994235,4cyclictest1001-21mta-sts-daemon10:06:181
671268994234,5cyclictest784252-21sed11:16:051
671268994232,6cyclictest420-21avahi-daemon12:16:171
671268994232,6cyclictest420-21avahi-daemon12:16:171
671268994230,9cyclictest420-21avahi-daemon12:35:441
671267994224,15cyclictest692245-21munin-run08:20:010
671269994137,2cyclictest0-21swapper/208:28:492
671269994131,4cyclictest0-21swapper/211:30:522
671269994130,8cyclictest785513-21sh11:20:012
671268994132,6cyclictest4066567-21cupsd09:29:461
671268994132,5cyclictest734655-21cron09:55:001
671267994133,5cyclictest713338-21ssh09:20:010
671269994032,5cyclictest725543-21munin-run09:40:012
671269994030,5cyclictest0-21swapper/209:45:462
671269994030,5cyclictest0-21swapper/209:45:462
671268994033,4cyclictest755845-21latency_hist10:30:021
671268994033,4cyclictest755845-21latency_hist10:30:021
671268994031,6cyclictest1-21systemd12:00:531
671268994031,5cyclictest1001-21mta-sts-daemon07:40:491
671268994031,5cyclictest1001-21mta-sts-daemon07:40:491
671267994032,5cyclictest737692-21sendmail10:00:010
671269993933,3cyclictest0-21swapper/210:15:012
671269993932,3cyclictest0-21swapper/210:40:012
671268993933,3cyclictest0-21swapper/109:40:011
671268993931,5cyclictest715082-21grep09:20:571
671268993931,5cyclictest704241-21munin-plugin-st09:00:001
671268993931,4cyclictest0-21swapper/109:00:481
671268993931,4cyclictest0-21swapper/109:00:481
671268993929,7cyclictest265-21systemd-journal09:31:481
671268993923,12cyclictest4066570-21cups-browsed10:12:571
671267993931,5cyclictest773600-21systemd-run11:00:010
671269993830,5cyclictest727216-21unixbench_multi09:40:562
671269993827,6cyclictest0-21swapper/209:25:482
671268993831,3cyclictest0-21swapper/112:10:471
671268993829,7cyclictest675766-21/usr/sbin/munin07:25:021
67126899382,33cyclictest798640-21ssh11:40:511
671267993831,4cyclictest828789-21threads12:30:540
671267993814,15cyclictest151ktimers/011:30:450
671269993731,3cyclictest701267-21kworker/u8:7-xprtiod09:35:012
671269993716,10cyclictest0-21swapper/209:20:482
67126899374,28cyclictest738955-21ssh10:00:571
671268993730,4cyclictest0-21swapper/110:31:281
671268993730,4cyclictest0-21swapper/110:31:281
671268993727,5cyclictest0-21swapper/112:30:471
671267993731,4cyclictest421-21dbus-daemon08:50:010
671267993723,10cyclictest151ktimers/010:20:490
67126799372,29cyclictest151ktimers/010:45:460
671267993718,12cyclictest151ktimers/012:05:520
671269993628,4cyclictest0-21swapper/212:16:032
671269993628,4cyclictest0-21swapper/212:16:032
671269993626,7cyclictest4066570-21cups-browsed12:10:022
671269993626,6cyclictest674292-21systemd-run07:20:022
671268993628,5cyclictest792742-21proc_pri11:30:521
671268993627,6cyclictest265-21systemd-journal10:41:181
671268993627,4cyclictest0-21swapper/109:40:451
671268993626,7cyclictest776584-21systemd11:05:021
671268993612,19cyclictest765766-21sshd10:45:481
671267993629,5cyclictest4066570-21cups-browsed11:17:020
671267993629,5cyclictest1000-21mta-sts-daemon11:05:520
671267993628,6cyclictest671266-21cyclictest12:05:020
671267993627,6cyclictest4066570-21cups-browsed11:36:290
671269993529,3cyclictest0-21swapper/209:15:492
671269993528,4cyclictest776546-21cron11:05:012
671269993527,5cyclictest265-21systemd-journal07:45:012
671269993527,5cyclictest265-21systemd-journal07:45:012
671269993524,8cyclictest735639-21missed_timers09:55:392
671268993526,4cyclictest0-21swapper/111:25:471
671268993525,7cyclictest683517-21aten_r2power_cu07:50:341
671267993529,4cyclictest740736-21systemd-run10:05:010
671267993528,4cyclictest749752-21latency_hist10:20:010
671269993428,3cyclictest0-21swapper/211:00:012
671269993427,4cyclictest809379-21cron12:00:012
67126999342,25cyclictest601-21gmain11:05:462
67126899343,16cyclictest786753-21ssh11:20:531
671268993426,5cyclictest780683-21ssh11:10:471
671268993425,6cyclictest4066570-21cups-browsed10:59:341
671268993424,6cyclictest421-21dbus-daemon07:20:021
671268993422,7cyclictest779639-21latency_hist11:10:031
671268993421,10cyclictest421-21dbus-daemon10:19:591
671267993427,4cyclictest799056-21timerwakeupswit11:41:000
671267993427,4cyclictest742152-21switchtime10:05:540
671267993427,4cyclictest718087-21unixbench_multi09:25:570
671267993426,6cyclictest421-21dbus-daemon09:10:010
671267993426,5cyclictest439-21systemd-logind11:50:010
671267993420,9cyclictest12650irq/32-eth011:00:510
67126999333,22cyclictest702207-21sshd08:50:482
671269993328,2cyclictest0-21swapper/210:29:532
671269993328,2cyclictest0-21swapper/210:29:532
671269993327,3cyclictest0-21swapper/211:40:372
671269993326,4cyclictest1003-21mta-sts-daemon10:43:342
671269993325,5cyclictest749759-21sendmail10:20:012
671269993325,3cyclictest0-21swapper/209:50:472
67126999330,25cyclictest750999-21ssh10:20:502
671268993327,3cyclictest0-21swapper/108:05:011
671268993323,7cyclictest420-21avahi-daemon11:57:121
671267993320,10cyclictest788480-21cron11:25:010
671269993226,3cyclictest0-21swapper/210:31:292
671269993226,3cyclictest0-21swapper/210:31:292
671269993224,5cyclictest421-21dbus-daemon12:40:012
67126999322,26cyclictest765743-21ssh10:45:472
671268993225,4cyclictest1003-21mta-sts-daemon12:26:061
671268993225,4cyclictest1003-21mta-sts-daemon12:26:061
671268993224,5cyclictest708658-21mailstats09:10:551
671268993224,5cyclictest696623-21sed08:30:551
671268993223,6cyclictest689766-21fschecks_count08:10:381
671268993222,6cyclictest677272-21systemd-run07:30:011
671267993228,2cyclictest0-21swapper/011:55:010
671267993226,3cyclictest0-21swapper/009:35:500
671267993223,6cyclictest676576-21needreboot07:25:430
671267993220,9cyclictest710304-21munin-plugin-st09:15:000
671267993220,9cyclictest0-21swapper/011:15:020
671267993217,11cyclictest420-21avahi-daemon09:30:520
671269993124,4cyclictest827482-21latency_hist12:30:022
671269993124,4cyclictest827482-21latency_hist12:30:022
671269993124,3cyclictest0-21swapper/211:20:412
671269993123,5cyclictest782509-21cron11:15:012
671269993123,5cyclictest682395-21kernelversion07:45:412
671269993122,6cyclictest800854-21aten_r2power_cu11:45:352
671269993121,7cyclictest4066570-21cups-browsed12:15:002
671268993124,4cyclictest0-21swapper/108:40:011
671268993123,5cyclictest695152-21unixbench_multi08:25:561
671268993121,7cyclictest420-21avahi-daemon07:30:431
671268993120,8cyclictest672741-21cron07:15:011
671267993126,3cyclictest1002-21mta-sts-daemon10:40:160
671267993125,4cyclictest816088-21fschecks_count12:10:390
671267993125,4cyclictest790114-21unixbench-2d11:25:570
671267993125,4cyclictest1003-21mta-sts-daemon09:40:460
671267993124,4cyclictest755832-21systemd10:30:020
671267993124,4cyclictest755832-21systemd10:30:020
671267993122,6cyclictest12650irq/32-eth009:50:510
671269993025,2cyclictest0-21swapper/209:00:012
671269993024,2cyclictest0-21swapper/210:05:362
671269993023,3cyclictest0-21swapper/210:00:502
671269993022,5cyclictest705762-21munin-plugin-st09:05:002
671269993022,5cyclictest705762-21munin-plugin-st09:05:002
671269993022,5cyclictest683895-21kernelversion07:50:402
671269993022,5cyclictest1-21systemd08:25:022
671269993021,6cyclictest691551-21needreboot08:15:422
67126999301,23cyclictest810633-21ssh12:00:512
671268993020,8cyclictest693868-21latency_hist08:25:021
671267993025,3cyclictest1000-21mta-sts-daemon09:00:530
671267993025,3cyclictest1000-21mta-sts-daemon09:00:530
671267993025,2cyclictest0-21swapper/012:20:010
671267993025,2cyclictest0-21swapper/012:20:010
671267993024,4cyclictest746737-21idleruntime-cro10:15:010
671267993024,4cyclictest689772-21fschecks_time08:10:380
671267993022,5cyclictest0-21swapper/010:50:450
671267993017,8cyclictest12650irq/32-eth009:20:460
671269992922,5cyclictest485-21in:imuxsock09:11:482
671269992921,5cyclictest679481-21/usr/sbin/munin07:35:412
671269992921,5cyclictest679481-21/usr/sbin/munin07:35:412
671269992919,7cyclictest822242-21latency12:20:412
671269992919,7cyclictest672655-21unixbench_multi07:10:592
67126899294,15cyclictest1003-21mta-sts-daemon07:45:461
671267992923,4cyclictest757381-21ssh10:31:350
671267992923,4cyclictest757381-21ssh10:31:350
671267992923,4cyclictest693692-21head08:20:550
67126799291,24cyclictest627-21lldpd11:58:220
671269992823,2cyclictest0-21swapper/211:30:022
671269992821,5cyclictest689760-21fschecks_count08:10:382
671269992820,5cyclictest4066570-21cups-browsed09:05:452
671269992819,6cyclictest677384-21latency_hist07:30:022
671268992820,5cyclictest702612-21grep08:50:551
671268992820,4cyclictest671328-21latency_hist07:10:021
671268992819,5cyclictest689254-21idleruntime-cro08:10:011
671267992822,4cyclictest0-21swapper/012:35:330
67126799282,22cyclictest729689-21ssh09:45:470
67126799282,22cyclictest729689-21ssh09:45:470
671267992822,2cyclictest0-21swapper/008:05:490
671269992722,2cyclictest0-21swapper/208:45:012
671269992720,3cyclictest0-21swapper/208:35:532
671267992721,4cyclictest696773-21systemd-run08:35:010
671267992721,4cyclictest687654-21sed08:01:010
67126999262,20cyclictest4066570-21cups-browsed07:55:502
671269992620,3cyclictest0-21swapper/208:01:012
671269992619,5cyclictest707254-21diskmemload11:35:162
67126999261,19cyclictest804498-21ssh11:50:452
671268992619,4cyclictest0-21swapper/109:05:401
671267992622,2cyclictest0-21swapper/008:40:010
671267992622,2cyclictest0-21swapper/007:45:480
671267992618,6cyclictest680781-21fschecks_count07:40:380
671267992618,6cyclictest680781-21fschecks_count07:40:380
671267992616,8cyclictest702768-21munin-run08:55:010
671269992519,4cyclictest701254-21awk08:50:012
671269992519,4cyclictest677695-21cpu07:30:352
671269992519,3cyclictest0-21swapper/207:20:482
671269992518,3cyclictest0-21swapper/208:30:512
67126899255,14cyclictest428-21mta-sts-daemon07:55:481
67126799254,17cyclictest12650irq/32-eth012:25:460
67126799254,17cyclictest12650irq/32-eth012:25:460
671267992516,6cyclictest151ktimers/008:40:440
671268992419,2cyclictest0-21swapper/107:40:001
671268992419,2cyclictest0-21swapper/107:40:001
671268992418,3cyclictest0-21swapper/108:40:571
671267992419,3cyclictest674793-21fschecks_count07:20:410
671267992418,4cyclictest680374-21latency_hist07:40:020
671267992418,4cyclictest680374-21latency_hist07:40:020
671267992416,5cyclictest0-21swapper/009:00:000
671268992312,7cyclictest701362-21latency_hist08:50:021
67126799238,10cyclictest151ktimers/007:10:470
671267992318,3cyclictest1002-21mta-sts-daemon07:50:420
67126799222,15cyclictest673248-21diskstats07:15:390
671267992215,5cyclictest0-21swapper/007:55:480
671269992115,3cyclictest0-21swapper/208:05:352
67127099202,7cyclictest0-21swapper/311:30:463
671269991914,2cyclictest0-21swapper/207:10:022
67127099182,6cyclictest0-21swapper/310:15:453
671267991814,2cyclictest671194-21kworker/u8:9-xprtiod07:10:020
67127099173,5cyclictest0-21swapper/311:05:473
67127099173,5cyclictest0-21swapper/310:20:453
67127099172,8cyclictest0-21swapper/310:50:523
67127099162,6cyclictest0-21swapper/311:00:503
67127099152,7cyclictest0-21swapper/308:28:483
67127099152,6cyclictest0-21swapper/311:50:493
67127099151,7cyclictest0-21swapper/310:55:473
67127099142,5cyclictest0-21swapper/308:10:453
67127099142,4cyclictest0-21swapper/307:10:463
67127099141,7cyclictest0-21swapper/312:10:463
67127099141,7cyclictest0-21swapper/309:15:453
67127099132,6cyclictest0-21swapper/312:30:463
67127099132,6cyclictest0-21swapper/309:55:483
67127099132,6cyclictest0-21swapper/309:50:403
67127099132,6cyclictest0-21swapper/309:30:523
67127099132,4cyclictest0-21swapper/308:20:423
67127099131,7cyclictest0-21swapper/311:35:543
67127099131,6cyclictest0-21swapper/312:25:473
67127099131,6cyclictest0-21swapper/312:25:473
67127099131,6cyclictest0-21swapper/310:45:503
67127099131,5cyclictest0-21swapper/307:25:453
67127099123,6cyclictest0-21swapper/308:35:013
67127099123,5cyclictest0-21swapper/311:55:483
67127099123,4cyclictest0-21swapper/312:00:463
67127099122,7cyclictest0-21swapper/310:10:463
67127099122,6cyclictest0-21swapper/311:40:473
67127099122,6cyclictest0-21swapper/309:25:443
67127099122,6cyclictest0-21swapper/309:00:473
67127099122,6cyclictest0-21swapper/309:00:473
67127099122,5cyclictest0-21swapper/312:20:443
67127099122,5cyclictest0-21swapper/308:50:483
67127099121,6cyclictest0-21swapper/311:10:503
67127099121,6cyclictest0-21swapper/310:05:453
67127099121,6cyclictest0-21swapper/309:10:443
67127099121,6cyclictest0-21swapper/307:30:483
67127099121,5cyclictest0-21swapper/307:55:463
67127099112,6cyclictest0-21swapper/309:45:473
67127099112,6cyclictest0-21swapper/309:45:473
67127099112,5cyclictest0-21swapper/312:05:383
67127099112,5cyclictest0-21swapper/311:45:403
67127099112,5cyclictest0-21swapper/308:35:473
67127099112,4cyclictest0-21swapper/308:55:413
67127099111,7cyclictest0-21swapper/312:35:493
67127099111,7cyclictest0-21swapper/308:00:473
67127099111,7cyclictest0-21swapper/307:15:463
67127099111,6cyclictest0-21swapper/312:15:383
67127099111,6cyclictest0-21swapper/312:15:383
67127099111,6cyclictest0-21swapper/311:25:353
67127099111,6cyclictest0-21swapper/311:20:373
67127099111,6cyclictest0-21swapper/311:15:483
67127099111,6cyclictest0-21swapper/310:35:363
67127099111,6cyclictest0-21swapper/310:34:263
67127099111,6cyclictest0-21swapper/310:34:263
67127099111,6cyclictest0-21swapper/309:35:453
67127099111,5cyclictest0-21swapper/309:05:463
67127099111,5cyclictest0-21swapper/307:35:563
67127099111,5cyclictest0-21swapper/307:35:563
67127099102,5cyclictest0-21swapper/310:00:423
67127099102,5cyclictest0-21swapper/309:20:333
67127099102,5cyclictest0-21swapper/308:40:423
67127099102,5cyclictest0-21swapper/308:05:503
67127099102,5cyclictest0-21swapper/307:50:483
67127099102,5cyclictest0-21swapper/307:45:473
67127099101,6cyclictest0-21swapper/310:25:403
67127099101,6cyclictest0-21swapper/310:25:403
67127099101,5cyclictest0-21swapper/310:40:363
67127099101,5cyclictest0-21swapper/309:40:333
6712709992,4cyclictest0-21swapper/307:20:413
6712709991,5cyclictest0-21swapper/308:45:423
6712709991,5cyclictest0-21swapper/308:15:363
6712709991,5cyclictest0-21swapper/307:40:433
6712709991,5cyclictest0-21swapper/307:40:433
6712709981,4cyclictest0-21swapper/307:10:033
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional