You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-12 - 13:53

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack5slot3s.osadl.org (updated Tue May 12, 2026 00:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
146142231162,56sleep00-21swapper/019:09:250
146142231162,56sleep00-21swapper/019:09:240
146592204167,25sleep10-21swapper/119:09:571
146592204167,25sleep10-21swapper/119:09:571
146492201165,24sleep30-21swapper/319:09:513
146492201165,24sleep30-21swapper/319:09:513
143972201164,24sleep20-21swapper/219:06:362
143972201164,24sleep20-21swapper/219:06:362
14755999692,3cyclictest0-21swapper/222:48:432
289182950,4sleep31476099cyclictest00:35:013
14755999491,2cyclictest0-21swapper/221:56:242
14755997673,2cyclictest0-21swapper/222:24:152
14755997470,3cyclictest0-21swapper/200:13:442
170582540,1sleep30-21swapper/300:05:233
273262510,1sleep00-21swapper/023:15:190
209222500,2sleep320921-21sort19:20:183
143022480,4sleep339-21rcuc/320:20:133
1475199461,37cyclictest16984-21awk00:05:211
1475199440,36cyclictest25894-21runrttasks23:12:591
1475199431,34cyclictest11065-21cat21:25:181
1475199431,34cyclictest11065-21cat21:25:181
1475199421,19cyclictest21001-21switchtime23:00:251
1475199411,28cyclictest7873-21basename21:20:011
14755993925,8cyclictest0-21swapper/222:12:412
1475199399,18cyclictest28695-21date19:40:011
1475199392,25cyclictest425-21/usr/sbin/munin23:30:121
1475199391,29cyclictest29443-21/usr/sbin/munin20:55:171
1475199391,27cyclictest28193-21smartctl00:30:261
1475199391,20cyclictest13325-21chrt23:58:101
1475199389,16cyclictest14190-21grep20:20:121
1475199381,29cyclictest32017-21taskset22:14:471
1475199381,28cyclictest28485-21/usr/sbin/munin23:20:131
1475199381,26cyclictest31645-21/usr/sbin/munin21:00:251
1475199381,24cyclictest16262-21/usr/sbin/munin20:25:181
1475199374,28cyclictest10015-40dnf21:24:401
1475199371,33cyclictest20774-21grep20:35:121
1475199371,32cyclictest808-21snmpd23:26:451
1475199371,32cyclictest1891-21sa119:50:201
1475199371,26cyclictest0-21swapper/122:05:121
1475199371,25cyclictest26290-21/usr/sbin/munin23:15:141
14743993711,12cyclictest30811-21awk19:45:020
1475199362,25cyclictest20336-21/usr/sbin/munin00:15:151
1475199361,32cyclictest26585-21memory22:00:201
1475199361,25cyclictest27246-21/usr/sbin/munin20:50:241
1475199361,23cyclictest18495-21grep19:15:151
1475199361,22cyclictest12840-21chrt21:30:141
1475199351,26cyclictest25147-21/usr/sbin/munin00:25:171
1475199351,22cyclictest808-21snmpd20:16:301
1475199351,22cyclictest808-21snmpd20:16:301
1475199351,20cyclictest10529-21/usr/sbin/munin23:50:231
1475199351,20cyclictest0-21swapper/122:35:161
14743993510,12cyclictest22013-21munin-run19:25:010
14755993420,9cyclictest0-21swapper/221:10:252
1475199341,30cyclictest1471-21munin-node21:05:121
1475199341,25cyclictest0-21swapper/100:35:241
1475199341,23cyclictest87550irq/128-enp1s0-20:45:241
1475199341,23cyclictest23310-21sort23:05:241
1475199341,21cyclictest3362-21mailstats22:20:221
1475199331,29cyclictest8035-21memory23:45:201
1475199331,29cyclictest8035-21memory23:45:201
1475199331,29cyclictest1921-21cat22:20:011
1475199331,29cyclictest15975-21hddtemp_smartct22:50:191
1475199331,29cyclictest15716-21awk00:05:011
1475199331,28cyclictest13033-21cat22:45:021
1475199331,21cyclictest5184-21latency22:25:201
1475199331,20cyclictest14949-21/usr/sbin/munin19:10:221
1475199331,19cyclictest13095-21/usr/sbin/munin22:45:231
1475199331,19cyclictest0-21swapper/121:40:171
1475199330,31cyclictest15621-21kworker/u8:020:40:231
1475199330,30cyclictest0-21swapper/121:45:231
1475199330,1cyclictest0-21swapper/100:25:031
1475199330,14cyclictest87650irq/129-enp1s0-20:31:371
1476099320,22cyclictest0-21swapper/300:19:353
1475599321,29cyclictest0-21swapper/223:22:412
1475599321,24cyclictest0-21swapper/219:14:382
1475599321,21cyclictest19475-21netstat20:30:202
1475599321,21cyclictest0-21swapper/223:40:132
1475599321,13cyclictest0-21swapper/222:58:412
1475199322,27cyclictest24945-21unixbench-2d21:55:251
1475199322,20cyclictest21273-21/usr/sbin/munin21:50:231
1475199322,17cyclictest7994-21/usr/sbin/munin22:30:251
1475199321,28cyclictest19136-21cut00:10:191
1475199321,27cyclictest6238-21sed20:00:191
1475199321,18cyclictest10836-21proc_pri20:10:241
1475199320,12cyclictest0-21swapper/121:13:151
1476099311,17cyclictest0-21swapper/321:56:313
1475599311,15cyclictest0-21swapper/223:45:552
1475599311,15cyclictest0-21swapper/223:45:552
1475599311,14cyclictest0-21swapper/219:38:312
1475199315,13cyclictest0-21swapper/123:35:161
1475199311,27cyclictest3315-21expr19:55:131
1475199311,27cyclictest20038-21/usr/sbin/munin19:20:191
1475199311,18cyclictest14677-21/usr/sbin/munin21:35:251
1475199311,17cyclictest18703-21processes22:55:221
1476099303,20cyclictest3460-21diskmemload21:24:423
1476099301,11cyclictest0-21swapper/300:20:393
1475599301,13cyclictest0-21swapper/219:44:072
1475199301,27cyclictest0-21swapper/120:06:251
1475199301,26cyclictest6985-21cat23:45:011
1475199301,26cyclictest31371-21df19:45:141
1475199301,26cyclictest29619-21/usr/sbin/munin19:40:171
1475199301,26cyclictest26400-21latency_hist19:35:011
14751993011,16cyclictest22239-21/usr/sbin/munin19:25:171
1474399307,11cyclictest13500-21perf00:00:020
14755992912,3cyclictest12232-21idleruntime23:55:172
1475599291,14cyclictest0-21swapper/221:47:132
1475599291,13cyclictest0-21swapper/222:35:232
1475599291,13cyclictest0-21swapper/221:01:452
1475599291,12cyclictest0-21swapper/220:15:572
1475599291,12cyclictest0-21swapper/220:15:572
1474399292,13cyclictest4699-21munin-run23:40:010
1476099281,18cyclictest0-21swapper/319:25:473
1476099281,12cyclictest0-21swapper/300:01:473
14755992817,3cyclictest0-21swapper/221:07:332
14755992812,7cyclictest0-21swapper/220:03:492
14755992812,3cyclictest0-21swapper/219:50:332
1475599281,17cyclictest12935-21sort22:45:012
1475599281,14cyclictest0-21swapper/200:20:132
1475599281,14cyclictest0-21swapper/200:09:232
1474399285,11cyclictest12245-21idleruntime-cro21:30:010
1474399285,11cyclictest12245-21idleruntime-cro21:30:010
1476099275,3cyclictest0-21swapper/322:16:113
1476099273,17cyclictest26402-40dnf00:26:253
1476099271,9cyclictest0-21swapper/320:00:553
1476099271,8cyclictest0-21swapper/320:18:153
1476099271,8cyclictest0-21swapper/320:18:153
1476099271,11cyclictest0-21swapper/323:50:113
1476099271,10cyclictest0-21swapper/321:47:313
1476099270,10cyclictest0-21swapper/320:26:273
14755992722,3cyclictest0-21swapper/220:51:142
14755992716,3cyclictest0-21swapper/219:20:372
14755992711,3cyclictest0-21swapper/221:18:092
1475599271,12cyclictest0-21swapper/220:24:372
14755992710,3cyclictest0-21swapper/200:19:212
1476099261,9cyclictest27565-21missed_timers19:35:193
1476099261,8cyclictest0-21swapper/319:55:553
1476099261,10cyclictest23726-21taskset23:08:193
1476099261,10cyclictest0-21swapper/320:40:433
1476099260,17cyclictest0-21swapper/300:36:233
1476099260,12cyclictest0-21swapper/322:48:433
1476099260,11cyclictest0-21swapper/323:17:233
1476099260,11cyclictest0-21swapper/321:16:403
1475599267,2cyclictest0-21swapper/220:40:262
1475599266,3cyclictest0-21swapper/220:29:182
1475599261,13cyclictest0-21swapper/223:31:572
1475599261,11cyclictest0-21swapper/222:00:172
1475599261,10cyclictest0-21swapper/221:44:532
1474399267,16cyclictest11290-21munin-run23:55:010
5362250,3sleep0541-21cpuspeed_turbos23:30:120
321522250,4sleep339-21rcuc/323:25:273
1476099251,9cyclictest0-21swapper/322:52:353
1476099251,9cyclictest0-21swapper/322:34:113
1476099251,10cyclictest0-21swapper/320:34:433
1476099250,9cyclictest0-21swapper/322:28:593
1476099250,10cyclictest0-21swapper/323:11:473
1476099250,10cyclictest0-21swapper/321:36:153
1476099250,10cyclictest0-21swapper/321:27:323
1476099250,10cyclictest0-21swapper/321:27:313
1476099250,10cyclictest0-21swapper/321:11:483
1476099250,10cyclictest0-21swapper/321:02:083
1475599259,3cyclictest0-21swapper/222:54:132
14755992513,3cyclictest0-21swapper/219:47:052
14755992512,2cyclictest0-21swapper/221:20:132
1475599251,14cyclictest0-21swapper/220:57:112
1475599251,11cyclictest0-21swapper/219:19:592
1475599251,10cyclictest0-21swapper/222:17:492
14755992510,3cyclictest0-21swapper/223:37:132
1474399257,15cyclictest2836-21munin-run19:55:010
1476099241,10cyclictest0-21swapper/323:00:273
1476099240,9cyclictest0-21swapper/319:19:443
1476099240,10cyclictest0-21swapper/323:38:243
1475599249,3cyclictest0-21swapper/223:51:012
1475599245,3cyclictest0-21swapper/223:09:422
1475599241,9cyclictest0-21swapper/220:06:572
1474399241,5cyclictest0-21swapper/022:09:420
1474399241,14cyclictest0-21swapper/022:20:140
1476099236,7cyclictest0-21swapper/322:14:093
1476099236,3cyclictest0-21swapper/322:39:273
1476099231,8cyclictest0-21swapper/319:46:193
1476099231,12cyclictest0-21swapper/321:08:203
1476099231,10cyclictest0-21swapper/323:23:073
1476099231,10cyclictest0-21swapper/300:14:233
1476099230,9cyclictest0-21swapper/320:35:233
1476099230,11cyclictest0-21swapper/322:42:113
1475599239,3cyclictest0-21swapper/219:31:572
14755992312,3cyclictest0-21swapper/220:46:542
1475599231,15cyclictest0-21swapper/200:37:412
1475599231,12cyclictest0-21swapper/221:25:172
1475599231,12cyclictest0-21swapper/221:25:172
1475599231,11cyclictest0-21swapper/220:39:062
1475599231,11cyclictest0-21swapper/219:29:452
14755992310,3cyclictest0-21swapper/221:54:012
1474399231,3cyclictest0-21swapper/000:35:220
1476099229,3cyclictest0-21swapper/323:40:283
1476099224,3cyclictest0-21swapper/319:42:033
1476099222,10cyclictest0-21swapper/321:30:223
1476099221,9cyclictest0-21swapper/320:50:193
1476099221,10cyclictest0-21swapper/319:11:033
1476099220,9cyclictest0-21swapper/321:51:433
1476099220,8cyclictest0-21swapper/319:31:433
1475599229,2cyclictest0-21swapper/200:33:332
1475599226,3cyclictest12825-21df_inode21:30:132
14755992212,3cyclictest0-21swapper/223:00:342
1475599221,10cyclictest14791-21ntp_states00:00:212
1474399221,3cyclictest17072-21irqstats20:25:180
245432211,2sleep024630-21proc_pri21:55:230
1476099214,5cyclictest2441-21df22:20:123
1476099211,12cyclictest0-21swapper/320:57:353
1476099210,7cyclictest0-21swapper/320:46:553
1475599218,3cyclictest0-21swapper/223:18:182
1475599218,3cyclictest0-21swapper/222:06:332
1475599217,3cyclictest0-21swapper/220:14:452
1475599216,3cyclictest0-21swapper/221:36:012
1475599211,12cyclictest3244-21cstates19:55:132
1474399211,6cyclictest0-21swapper/021:36:200
1474399211,6cyclictest0-21swapper/000:03:180
1474399211,4cyclictest0-21swapper/022:37:260
1476099207,3cyclictest0-21swapper/322:00:513
1476099202,3cyclictest0-21swapper/323:34:483
1476099201,7cyclictest0-21swapper/320:08:353
1476099200,7cyclictest0-21swapper/322:56:033
1476099200,5cyclictest0-21swapper/321:43:433
1475599207,3cyclictest0-21swapper/222:27:132
1474399205,9cyclictest0-21swapper/020:45:110
1474399201,7cyclictest0-21swapper/022:50:290
1474399201,4cyclictest7-21ksoftirqd/023:10:150
14743992013,4cyclictest14707-21cut20:20:160
1474399201,10cyclictest27227-21head22:00:290
1474399201,10cyclictest25719-21needreboot00:25:210
14743992010,3cyclictest87550irq/128-enp1s0-23:00:100
139282200,3sleep00-21swapper/022:45:170
1476099194,6cyclictest0-21swapper/323:46:413
1476099194,6cyclictest0-21swapper/323:46:413
1476099191,11cyclictest3460-21diskmemload22:06:143
1476099191,10cyclictest0-21swapper/323:58:163
1475599197,3cyclictest0-21swapper/223:28:492
1475599191,8cyclictest0-21swapper/200:26:242
1475599190,12cyclictest0-21swapper/222:31:352
1474399198,3cyclictest18139-21/usr/sbin/munin00:10:140
1474399197,7cyclictest0-21swapper/019:49:120
1474399195,8cyclictest0-21swapper/020:42:090
1474399194,13cyclictest11-21rcuc/023:40:150
1474399190,12cyclictest0-21swapper/019:35:180
1476099181,10cyclictest15621-21kworker/u8:019:50:103
1475599180,11cyclictest0-21swapper/223:10:222
1474399188,6cyclictest0-21swapper/022:29:570
1474399183,3cyclictest7-21ksoftirqd/000:05:170
1474399182,9cyclictest11-21rcuc/020:15:130
1474399182,9cyclictest11-21rcuc/020:15:130
1474399182,15cyclictest4055-21kworker/0:322:43:140
1474399181,8cyclictest0-21swapper/000:22:100
1474399181,7cyclictest0-21swapper/021:45:220
1474399181,7cyclictest0-21swapper/021:18:100
1474399181,7cyclictest0-21swapper/020:01:080
1474399181,6cyclictest0-21swapper/023:47:150
1474399181,6cyclictest0-21swapper/023:47:150
1474399181,6cyclictest0-21swapper/022:30:340
1474399181,5cyclictest30117-21hddtemp_smartct20:55:160
1474399181,4cyclictest0-21swapper/022:17:030
1474399181,10cyclictest28303-21ntpq20:50:220
1474399181,10cyclictest0-21swapper/021:41:140
302512171,2sleep00-21swapper/023:22:310
1476099171,6cyclictest9662-21/usr/sbin/munin20:10:123
1474399178,3cyclictest0-21swapper/021:10:270
1474399175,4cyclictest3460-21diskmemload23:28:330
1474399174,11cyclictest11-21rcuc/020:05:120
1474399171,7cyclictest1471-21/usr/sbin/munin21:05:150
1474399171,7cyclictest0-21swapper/020:30:130
1474399171,6cyclictest0-21swapper/000:15:200
1474399171,5cyclictest0-21swapper/021:50:240
1474399171,3cyclictest26932-21/usr/sbin/munin00:30:250
1474399171,14cyclictest81ktimersoftd/020:10:140
14743991711,4cyclictest488-21/usr/sbin/munin21:00:240
14743991711,3cyclictest827-21runrttasks22:12:140
1474399171,10cyclictest0-21swapper/019:30:260
100242171,2sleep00-21swapper/021:24:440
166022161,2sleep00-21swapper/019:11:040
14743991612,3cyclictest0-21swapper/020:35:230
14743991611,3cyclictest3062-21/usr/sbin/munin19:55:190
14743991611,3cyclictest23485-21mailstats19:25:230
14743991611,3cyclictest22944-21ntp_kernel_pll_23:05:200
14743991611,3cyclictest13516-21ntp_kernel_pll_21:30:190
14743991610,4cyclictest0-21swapper/022:55:180
1474399160,6cyclictest17907-21cpuspeed_turbos19:15:110
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional