You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-20 - 03:09

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Thu Mar 19, 2026 12:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
790875997466,7cyclictest75-21kswapd011:11:240
790885995552,2cyclictest75-21kswapd011:11:233
790883994440,3cyclictest75-21kswapd011:11:232
790883994035,3cyclictest966705-21kworker/u8:0+events_unbound10:20:162
790883993731,4cyclictest1033908-21/usr/sbin/munin11:05:132
790885993631,3cyclictest972064-21/usr/sbin/munin10:05:133
790883993630,5cyclictest1122261-21/usr/sbin/munin12:30:132
790875993623,3cyclictest915028-21apt-get09:10:100
790878993025,3cyclictest790867-21kworker/u8:0+flush-259:007:20:151
790883992923,4cyclictest252-21systemd-journal10:15:002
790878992923,4cyclictest1096378-21apt-get12:05:091
790883992823,3cyclictest1039041-21kworker/u8:1+events_unbound11:30:152
790875992824,3cyclictest1096137-21kworker/u8:3+flush-259:012:04:590
790875992722,4cyclictest966705-21kworker/u8:0+flush-259:011:00:170
790878992621,4cyclictest1064703-21apt-get11:35:011
790875992621,3cyclictest961121-21kworker/u8:2+flush-259:010:35:150
790885992520,4cyclictest1096137-21kworker/u8:3+flush-259:012:15:153
790885992519,4cyclictest1085792-21latency_hist11:55:023
790883992521,3cyclictest1096137-21kworker/u8:3+flush-259:012:30:002
790878992520,3cyclictest796256-21apt-get07:15:121
790885992420,3cyclictest1054373-21kworker/u8:0+flush-259:011:35:003
790885992419,3cyclictest252-21systemd-journal11:18:453
790883992419,3cyclictest868528-21kworker/u8:3+flush-259:008:55:182
790883992418,4cyclictest935920-21cat09:30:012
790883992416,5cyclictest795991-21latency_hist07:15:002
790885992319,3cyclictest868528-21kworker/u8:3+flush-259:008:50:153
790885992316,5cyclictest252-21systemd-journal12:35:003
790878992317,4cyclictest919929-21kworker/u8:1+flush-259:009:45:181
790885992217,3cyclictest801396-21apt-get07:20:113
790883992219,2cyclictest1075482-21kworker/u8:2+events_unbound12:00:132
790883992216,4cyclictest873876-21/usr/sbin/munin08:30:132
790885992116,4cyclictest1075882-21diskstats11:45:123
790885992115,4cyclictest252-21systemd-journal11:40:013
790885992114,5cyclictest1054354-21latency_hist11:25:013
790883992118,2cyclictest1049181-21sh11:20:002
790883992117,3cyclictest966705-21kworker/u8:0+flush-259:010:20:002
790883992115,4cyclictest956418-21latency_hist09:50:012
790883992114,5cyclictest805469-21ntpq07:20:212
790883992114,5cyclictest1018190-21latency_hist10:50:002
790878992118,2cyclictest832-21runrttasks08:05:011
790878992118,2cyclictest832-21runrttasks08:05:001
790878992115,4cyclictest1111651-21latency_hist12:20:001
790885992016,3cyclictest1096137-21kworker/u8:3+flush-259:012:24:593
790883992017,2cyclictest966705-21kworker/u8:0+flush-259:011:00:152
790878992013,6cyclictest981240-21ntp_kernel_pll_10:10:201
790885991915,3cyclictest919929-21kworker/u8:1+flush-259:009:55:163
790885991912,5cyclictest904486-21latency_hist09:00:023
790883991915,3cyclictest842812-21kworker/u8:0+flush-259:008:10:002
790883991914,3cyclictest868528-21kworker/u8:3+flush-259:009:25:242
79087899191,17cyclictest0-21swapper/111:15:001
79087899190,10cyclictest1017874-21taskset10:46:101
790875991916,2cyclictest795984-21sh07:15:000
790875991915,3cyclictest919929-21kworker/u8:1+flush-259:009:25:170
790875991914,3cyclictest801132-21latency_hist07:20:000
790885991815,2cyclictest0-21swapper/308:45:003
790885991811,5cyclictest252-21systemd-journal07:50:003
790883991815,2cyclictest847913-21cron08:05:002
790883991815,2cyclictest847913-21cron08:05:002
790883991812,4cyclictest252-21systemd-journal10:08:462
790883991811,5cyclictest252-21systemd-journal11:35:462
79087899185,6cyclictest0-21swapper/109:17:241
79087899180,9cyclictest858028-21chrt08:13:021
79087899180,9cyclictest1070013-21chrt11:36:331
79087899180,9cyclictest1054052-21chrt11:21:311
79087899180,9cyclictest1038654-21chrt11:08:171
79087899180,9cyclictest1007528-21seq10:35:511
79087899180,10cyclictest966347-21taskset09:56:121
790875991813,4cyclictest878993-21/usr/sbin/munin08:35:140
790875991813,4cyclictest806292-21latency_hist07:25:000
790885991713,2cyclictest910938-21kthreadcore09:05:183
790885991712,3cyclictest1-21systemd10:20:223
790885991710,5cyclictest1002719-21latency_hist10:35:003
790883991714,2cyclictest1080620-21idleruntime-cro11:50:012
79087899173,12cyclictest993724-21cut10:25:171
79087899173,12cyclictest993724-21cut10:25:161
79087899173,12cyclictest824565-21tr07:40:171
79087899171,7cyclictest2109693-21snmpd10:22:131
790878991713,3cyclictest2925013-21kworker/u8:4+flush-259:007:10:281
79087899170,9cyclictest1095907-21seq12:02:381
79087899170,8cyclictest986934-21taskset10:16:421
79087899170,10cyclictest1012804-21chrt10:41:441
790875991714,2cyclictest60350irq/132-enp0s29f1:rx-009:15:330
790875991712,3cyclictest1059971-21diskstats11:30:120
790885991611,4cyclictest507-21in:imuxsock10:20:013
790883991611,4cyclictest868528-21kworker/u8:3+events_unbound08:45:132
79087899167,7cyclictest0-21swapper/112:15:001
79087899165,9cyclictest884343-21entropy08:40:131
79087899163,11cyclictest1056614-21cut11:25:171
790878991610,4cyclictest935425-21cp09:30:011
79087899160,9cyclictest883652-21taskset08:36:571
79087899160,9cyclictest1028317-21seq10:57:421
79087899160,8cyclictest1132224-21seq12:38:001
79087899160,2cyclictest868682-21cat08:25:011
79087899160,14cyclictest937789-21grep09:30:171
79087899160,13cyclictest1075811-21tr11:45:131
79087899160,12cyclictest858573-21cpuspeed_turbos08:15:121
790875991614,1cyclictest60350irq/132-enp0s29f1:rx-011:18:460
790875991614,1cyclictest60350irq/132-enp0s29f1:rx-009:30:210
790875991613,2cyclictest60350irq/132-enp0s29f1:rx-012:20:480
790875991613,2cyclictest60350irq/132-enp0s29f1:rx-011:22:020
790875991613,2cyclictest60350irq/132-enp0s29f1:rx-010:50:190
790875991613,2cyclictest60350irq/132-enp0s29f1:rx-007:55:200
790875991611,4cyclictest1065216-21base6411:35:010
79088599159,4cyclictest858634-21diskstats08:15:133
790885991512,2cyclictest1127980-21irqcore12:35:143
790885991511,3cyclictest936008-21apt-get09:30:103
790885991511,3cyclictest868528-21kworker/u8:3+flush-259:009:13:163
790885991510,3cyclictest806956-21/usr/sbin/munin07:25:153
79088399159,4cyclictest252-21systemd-journal12:13:452
790883991513,1cyclictest911996-21kthreadcore09:05:192
790883991511,3cyclictest837365-21kworker/u8:2+flush-259:008:50:212
790883991511,3cyclictest790867-21kworker/u8:0+events_unbound07:35:142
790883991511,3cyclictest1075482-21kworker/u8:2+flush-259:011:55:002
790883991511,2cyclictest503-21tpm2-abrmd10:05:012
790883991511,2cyclictest0-21swapper/209:15:162
79087899154,9cyclictest252-21systemd-journal09:22:461
79087899152,11cyclictest790959-21cat07:10:011
79087899152,11cyclictest1081066-21tune2fs11:50:131
79087899151,12cyclictest869073-21grep08:25:141
79087899151,12cyclictest842772-21latency_hist08:00:001
79087899151,12cyclictest252-21systemd-journal09:15:001
79087899151,12cyclictest252-21systemd-journal08:50:001
79087899151,12cyclictest1122696-21irqcore12:30:141
79087899150,13cyclictest910341-21cut09:05:171
79087899150,13cyclictest899341-21sh08:55:011
79087899150,13cyclictest878742-21apt08:35:001
79087899150,13cyclictest807094-21grep07:25:141
79087899150,13cyclictest1070974-21sed11:40:131
79087899150,12cyclictest972473-21/usr/sbin/munin10:05:151
79087899150,12cyclictest946243-21grep09:40:001
79087899150,12cyclictest848339-21tr08:05:121
79087899150,12cyclictest1028877-21grep11:00:121
790875991514,1cyclictest60350irq/132-enp0s29f1:rx-010:20:420
790875991514,1cyclictest60350irq/132-enp0s29f1:rx-008:56:520
790875991513,2cyclictest60350irq/132-enp0s29f1:rx-008:09:060
790875991513,1cyclictest60350irq/132-enp0s29f1:rx-011:49:310
790875991513,1cyclictest60350irq/132-enp0s29f1:rx-010:05:040
790875991513,1cyclictest60350irq/132-enp0s29f1:rx-010:02:240
790875991513,1cyclictest60350irq/132-enp0s29f1:rx-008:45:110
790875991513,1cyclictest60350irq/132-enp0s29f1:rx-008:30:400
790875991513,1cyclictest60350irq/132-enp0s29f1:rx-008:10:150
790875991512,2cyclictest60350irq/132-enp0s29f1:rx-012:25:230
790875991512,2cyclictest60350irq/132-enp0s29f1:rx-011:27:440
790875991512,2cyclictest60350irq/132-enp0s29f1:rx-010:41:580
790875991512,2cyclictest60350irq/132-enp0s29f1:rx-010:30:230
790875991512,2cyclictest60350irq/132-enp0s29f1:rx-010:10:110
790875991512,2cyclictest60350irq/132-enp0s29f1:rx-010:00:010
790875991512,2cyclictest0-21swapper/009:49:590
790875991511,3cyclictest1111879-21kworker/u8:0+flush-259:012:35:160
790875991511,3cyclictest1096137-21kworker/u8:3+flush-259:012:15:160
790875991511,3cyclictest1039041-21kworker/u8:1+flush-259:012:15:000
79088599147,5cyclictest252-21systemd-journal08:35:003
790885991411,2cyclictest2925013-21kworker/u8:4+events_unbound07:10:183
790885991410,3cyclictest966705-21kworker/u8:0+flush-259:010:55:013
790885991410,3cyclictest966705-21kworker/u8:0+flush-259:010:45:243
790885991410,3cyclictest863655-21apt-get08:20:103
79088399149,3cyclictest1107214-21irqcore12:15:152
79088399148,4cyclictest811515-21kworker/u8:1+flush-259:007:45:172
790883991411,2cyclictest916048-21kthreadcore09:10:162
790883991410,3cyclictest879277-21fschecks_time08:35:142
790883991410,3cyclictest807155-21fschecks_count07:25:152
79087899141,11cyclictest946783-21cut09:40:161
79087899141,11cyclictest904898-21tr09:00:121
79087899141,11cyclictest903533-21cut08:55:211
79087899141,11cyclictest827714-21cpuspeed_turbos07:45:141
79087899141,11cyclictest812242-21grep07:30:111
79087899141,11cyclictest1117493-21ps12:25:151
79087899141,11cyclictest1089255-21kthreadcore11:55:171
79087899141,10cyclictest1002858-21latency_hist10:35:001
79087899140,12cyclictest971402-21grep10:00:251
79087899140,12cyclictest958938-21sed09:50:161
79087899140,12cyclictest837626-21ls07:55:001
79087899140,12cyclictest822169-21latency_hist07:40:001
79087899140,11cyclictest1044612-21/usr/sbin/munin11:15:161
790875991413,1cyclictest60350irq/132-enp0s29f1:rx-010:27:090
790875991413,1cyclictest60350irq/132-enp0s29f1:rx-010:27:090
790875991413,1cyclictest60350irq/132-enp0s29f1:rx-009:00:200
790875991413,1cyclictest60350irq/132-enp0s29f1:rx-008:15:240
790875991412,1cyclictest827361-21turbostat07:45:000
790875991412,1cyclictest60350irq/132-enp0s29f1:rx-010:45:230
790875991412,1cyclictest60350irq/132-enp0s29f1:rx-009:40:120
790875991412,1cyclictest60350irq/132-enp0s29f1:rx-009:35:220
790875991412,1cyclictest60350irq/132-enp0s29f1:rx-008:50:350
790875991412,1cyclictest60350irq/132-enp0s29f1:rx-008:42:270
790875991412,1cyclictest60350irq/132-enp0s29f1:rx-007:50:110
790875991412,1cyclictest60350irq/132-enp0s29f1:rx-007:45:150
790875991412,1cyclictest60350irq/132-enp0s29f1:rx-007:30:220
790875991411,2cyclictest61550irq/141-enp0s29f1:tx-409:20:160
790875991411,2cyclictest60350irq/132-enp0s29f1:rx-012:05:210
790875991411,2cyclictest60350irq/132-enp0s29f1:rx-008:22:590
79088599139,3cyclictest1054373-21kworker/u8:0+flush-259:011:41:503
79088599139,2cyclictest0-21swapper/311:10:013
79088599138,4cyclictest961121-21kworker/u8:2+flush-259:010:00:183
79088599138,3cyclictest1018239-21kworker/u8:3+flush-259:010:55:233
790885991311,1cyclictest818000-21irqcore07:35:163
790885991310,2cyclictest981136-21needreboot10:10:173
790885991310,2cyclictest951798-21fschecks_time09:45:153
790885991310,2cyclictest868528-21kworker/u8:3+flush-259:009:01:073
790885991310,2cyclictest1091269-21cpuspeed_turbos12:00:133
790885991310,2cyclictest1028758-21/usr/sbin/munin11:00:223
79088399139,3cyclictest987261-21kworker/u8:1+flush-259:010:50:222
79088399139,3cyclictest796513-21fschecks_time07:15:152
79088399138,3cyclictest868528-21kworker/u8:3+flush-259:009:20:192
79088399138,3cyclictest790867-21kworker/u8:0+flush-259:007:30:142
79088399136,4cyclictest252-21systemd-journal09:54:592
79088399131,3cyclictest842764-21awk08:00:002
790883991310,2cyclictest951260-21cron09:44:592
790883991310,2cyclictest811515-21kworker/u8:1+flush-259:007:40:222
790883991310,2cyclictest1097398-21tr12:05:142
790883991310,2cyclictest1075482-21kworker/u8:2+events_unbound11:55:152
79087899130,4cyclictest1018545-21expr10:50:141
790875991312,1cyclictest60350irq/132-enp0s29f1:rx-011:09:420
790875991310,2cyclictest60350irq/132-enp0s29f1:rx-011:55:260
790875991310,2cyclictest60350irq/132-enp0s29f1:rx-011:50:120
790875991310,2cyclictest60350irq/132-enp0s29f1:rx-009:50:110
790875991310,2cyclictest60350irq/132-enp0s29f1:rx-008:00:150
790875991310,2cyclictest60350irq/132-enp0s29f1:rx-008:00:150
790875991310,2cyclictest252-21systemd-journal08:30:000
790875991310,2cyclictest1070788-21fschecks_time11:40:120
79088599128,3cyclictest1075482-21kworker/u8:2+flush-259:012:07:173
79088599127,4cyclictest868528-21kworker/u8:3+flush-259:009:20:263
79088599127,4cyclictest842812-21kworker/u8:0+flush-259:008:25:163
79088599127,3cyclictest951270-21basename09:44:593
79088599121,9cyclictest1122031-21awk12:29:593
79088399128,2cyclictest0-21swapper/211:25:172
79088399126,4cyclictest1013087-21latency_hist10:45:002
79087599129,2cyclictest60350irq/132-enp0s29f1:rx-009:05:130
790875991210,1cyclictest60350irq/132-enp0s29f1:rx-012:32:230
79088599119,1cyclictest0-21swapper/309:30:013
79088599118,2cyclictest1010458-21kthreadcore10:40:163
79088599117,3cyclictest837365-21kworker/u8:2+flush-259:007:55:263
79088599116,3cyclictest1054589-21/usr/sbin/munin11:25:143
79088399118,2cyclictest998003-21fschecks_count10:30:132
79088399118,2cyclictest832852-21cpuspeed_turbos07:50:122
79088399118,2cyclictest1070632-21cpuspeed_turbos11:40:102
79088399118,2cyclictest0-21swapper/209:59:592
79088399118,1cyclictest0-21swapper/210:55:132
79088399117,1cyclictest858474-21sshd08:15:112
79087599117,3cyclictest60350irq/132-enp0s29f1:rx-007:35:120
79088599105,3cyclictest893199-21sshd08:45:193
79088399107,2cyclictest1007863-21journal-offline10:40:012
79087599106,1cyclictest60350irq/132-enp0s29f1:rx-010:15:230
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional