You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-20 - 21:21

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Mon Apr 20, 2026 12:44:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
152052246158,39sleep20-21swapper/207:09:362
149542240166,24sleep10-21swapper/107:06:221
150022201164,24sleep30-21swapper/307:07:003
149892201163,26sleep00-21swapper/007:06:500
2690721440,3sleep11532099cyclictest10:00:141
15325996865,2cyclictest0-21swapper/209:27:432
295752590,3sleep21532599cyclictest07:40:132
69772580,3sleep00-21swapper/008:00:220
178642500,3sleep11532099cyclictest08:25:191
88682410,1sleep20-21swapper/211:45:232
1532599371,13cyclictest4025-21diskmemload10:07:022
87162330,2sleep01530999cyclictest09:20:130
1530999339,11cyclictest23802-21idleruntime-cro09:55:010
1530999325,12cyclictest28226-21awk10:05:010
1532599311,11cyclictest0-21swapper/208:23:462
1530999313,13cyclictest32609-21munin-run10:15:010
1532599301,13cyclictest0-21swapper/207:40:012
1532599291,12cyclictest0-21swapper/211:38:582
1532599291,11cyclictest0-21swapper/209:04:012
1532599291,10cyclictest0-21swapper/207:57:462
1532599290,12cyclictest0-21swapper/207:54:542
1530999291,13cyclictest18467-21munin-run12:10:010
1530999291,13cyclictest18467-21munin-run12:10:010
1532599281,11cyclictest0-21swapper/212:09:142
1532599281,11cyclictest0-21swapper/212:09:142
1532599280,12cyclictest0-21swapper/212:24:012
1533099278,6cyclictest18044-40dnf08:25:253
1532599271,11cyclictest11303-21ntp_states08:10:222
1532599271,11cyclictest0-21swapper/209:06:292
1532599271,10cyclictest0-21swapper/211:16:372
1532599271,10cyclictest0-21swapper/208:53:492
1532599270,12cyclictest0-21swapper/210:51:572
1530999271,13cyclictest31371-21munin-run07:45:010
1532599268,10cyclictest15349-21ntp_offset12:00:222
1532599261,11cyclictest0-21swapper/207:25:052
1532599261,10cyclictest0-21swapper/211:44:302
1532599261,10cyclictest0-21swapper/210:34:092
1532599261,10cyclictest0-21swapper/209:59:382
1532599260,2cyclictest0-21swapper/211:25:292
1532599260,12cyclictest0-21swapper/210:41:532
1532599260,10cyclictest0-21swapper/207:33:292
15320992620,3cyclictest20666-21munin-run12:15:011
1530999267,16cyclictest826-21munin-run11:30:010
1530999261,11cyclictest9200-21grep11:45:250
1530999260,10cyclictest5869-21needreboot10:25:200
167592251,20sleep016845-21needreboot10:50:200
1533099253,18cyclictest27037-40dnf12:27:243
1532599251,9cyclictest4025-21diskmemload10:48:132
1532599251,9cyclictest0-21swapper/212:27:492
1532599251,9cyclictest0-21swapper/211:03:172
1532599251,4cyclictest0-21swapper/209:52:532
1532599251,13cyclictest0-21swapper/211:30:142
1532599251,10cyclictest0-21swapper/208:43:132
1532599251,10cyclictest0-21swapper/208:17:262
1532599250,9cyclictest0-21swapper/209:15:492
1532599250,8cyclictest0-21swapper/210:55:132
1532599250,8cyclictest0-21swapper/210:55:132
1532599250,10cyclictest0-21swapper/209:14:012
1532599250,10cyclictest0-21swapper/208:32:062
1532599250,10cyclictest0-21swapper/207:23:092
1532099254,3cyclictest21-21rcuc/107:54:061
15320992516,2cyclictest23061-21awk09:50:231
1530999251,10cyclictest3050-21strings09:05:200
1530999250,12cyclictest30423-21awk10:10:010
1533099243,9cyclictest0-21swapper/311:37:033
1533099241,11cyclictest28398-21memory12:30:203
1533099240,6cyclictest0-21swapper/311:03:013
1532599241,9cyclictest0-21swapper/207:49:062
1532599241,9cyclictest0-21swapper/207:15:092
1532599241,10cyclictest0-21swapper/212:37:332
1532599241,10cyclictest0-21swapper/212:11:012
1532599240,9cyclictest0-21swapper/209:23:052
15320992417,4cyclictest5176-21memory09:10:201
1530999241,10cyclictest10265-21grep10:35:210
1533099234,10cyclictest18535-21grep10:55:153
1533099234,10cyclictest18535-21grep10:55:153
1533099231,6cyclictest0-21swapper/312:03:143
1533099231,6cyclictest0-21swapper/311:49:443
1533099231,6cyclictest0-21swapper/311:44:243
1533099231,13cyclictest0-21swapper/311:05:163
1533099230,6cyclictest0-21swapper/312:05:423
1533099230,6cyclictest0-21swapper/312:05:423
1533099230,11cyclictest0-21swapper/310:05:293
1532599237,2cyclictest0-21swapper/209:48:062
1532599237,2cyclictest0-21swapper/209:48:062
1532599231,9cyclictest0-21swapper/209:35:462
1532599231,9cyclictest0-21swapper/208:49:572
1532599231,10cyclictest0-21swapper/211:51:302
1532599231,10cyclictest0-21swapper/208:25:262
1532599230,8cyclictest0-21swapper/210:25:212
1532599230,11cyclictest0-21swapper/208:39:332
1532599230,10cyclictest0-21swapper/210:15:342
1532599230,10cyclictest0-21swapper/209:41:022
1532599230,10cyclictest0-21swapper/208:07:182
1532599230,10cyclictest0-21swapper/207:13:182
1532099239,4cyclictest29750-21diskstats07:40:161
15320992317,3cyclictest21324-21ntp_states11:00:181
15320992317,3cyclictest20954-21df_inode07:20:131
15320992317,3cyclictest18756-21df_inode07:15:131
1532099231,4cyclictest10530-21expr08:10:121
1532099231,4cyclictest0-21swapper/108:30:121
1532099231,19cyclictest27550-21diskstats07:35:151
1532099231,19cyclictest0-21swapper/110:42:331
1532099230,5cyclictest0-21swapper/110:26:231
1530999237,5cyclictest28548-21NetworkManager11:16:300
15309992314,5cyclictest18397-21grep08:25:250
1530999231,11cyclictest32207-21/usr/sbin/munin09:00:160
1530999231,11cyclictest10235-21/usr/sbin/munin08:10:220
1530999230,7cyclictest0-21swapper/011:10:210
1533099223,3cyclictest0-21swapper/310:00:593
1533099221,4cyclictest0-21swapper/310:50:223
1533099221,12cyclictest0-21swapper/312:35:033
1533099220,9cyclictest0-21swapper/311:29:273
1533099220,9cyclictest0-21swapper/309:31:393
1533099220,9cyclictest0-21swapper/309:22:333
1533099220,11cyclictest0-21swapper/311:14:473
1533099220,10cyclictest0-21swapper/309:15:273
1532599229,2cyclictest0-21swapper/208:03:282
1532599227,3cyclictest0-21swapper/211:10:332
1532599221,6cyclictest0-21swapper/210:24:202
1532599220,9cyclictest0-21swapper/211:06:532
1532599220,9cyclictest0-21swapper/210:13:422
1532599220,9cyclictest0-21swapper/208:59:212
1532599220,10cyclictest0-21swapper/212:18:172
1532099221,15cyclictest0-21swapper/110:59:291
1532099221,15cyclictest0-21swapper/110:59:291
1530999229,2cyclictest0-21swapper/008:20:130
1530999227,4cyclictest20909-21sshd08:34:050
1530999226,3cyclictest16096-21kernelversion09:35:190
1530999226,14cyclictest11-21rcuc/011:35:010
1530999221,6cyclictest27900-21idleruntime07:35:170
1530999221,10cyclictest31740-21sshd08:57:050
251722210,4sleep011-21rcuc/012:25:010
1533099219,4cyclictest0-21swapper/309:46:073
1533099219,4cyclictest0-21swapper/309:46:073
1533099215,9cyclictest4998-21timerandwakeup07:55:243
1533099214,9cyclictest0-21swapper/307:21:053
1533099211,4cyclictest0-21swapper/311:51:043
1533099211,4cyclictest0-21swapper/311:25:013
1533099211,12cyclictest4025-21diskmemload09:10:133
1533099211,11cyclictest579-21systemd-journal11:35:013
1533099210,8cyclictest0-21swapper/310:36:013
1532599216,4cyclictest27823-21users10:00:262
1532599210,9cyclictest0-21swapper/210:39:172
1532599210,10cyclictest0-21swapper/209:33:022
1532099218,7cyclictest0-21swapper/110:53:071
1532099218,11cyclictest23-21ksoftirqd/108:00:161
1532099211,8cyclictest29680-21/usr/sbin/munin12:35:201
1532099211,2cyclictest116312chrt11:52:111
15320992111,8cyclictest27679-21kernelversion11:15:191
1532099211,18cyclictest0-21swapper/111:29:531
1532099211,18cyclictest0-21swapper/110:35:201
1530999213,15cyclictest2037-21gltestperf07:50:150
1530999211,10cyclictest2127-21sshd10:16:040
225722200,2sleep30-21swapper/312:16:313
190862200,2sleep00-21swapper/009:40:560
153552200,4sleep01530999cyclictest12:00:210
1533099206,3cyclictest0-21swapper/309:25:133
1533099203,4cyclictest10011-21perf08:10:013
1533099201,4cyclictest0-21swapper/311:57:243
1533099201,4cyclictest0-21swapper/309:55:323
1533099201,11cyclictest365-21/usr/sbin/munin10:15:183
1533099201,11cyclictest0-21swapper/312:12:063
1533099201,11cyclictest0-21swapper/310:45:333
1533099201,11cyclictest0-21swapper/310:42:103
1533099201,11cyclictest0-21swapper/310:31:163
1533099201,11cyclictest0-21swapper/310:26:433
1533099201,11cyclictest0-21swapper/310:11:113
1533099200,9cyclictest0-21swapper/312:20:283
1532599206,3cyclictest0-21swapper/211:23:412
1532099206,3cyclictest0-21swapper/107:45:141
1532099205,6cyclictest0-21swapper/110:08:171
1532099205,13cyclictest9659-21kworker/u8:109:46:101
1532099205,13cyclictest9659-21kworker/u8:109:46:101
1532099204,3cyclictest29367-21sshd12:34:051
1532099201,7cyclictest3632-21/usr/sbin/munin07:55:181
15320992016,3cyclictest0-21swapper/112:00:201
15320992016,3cyclictest0-21swapper/110:10:191
15320992014,3cyclictest2710-21grep09:05:161
1532099201,2cyclictest25648-21/usr/sbin/munin12:25:141
1532099201,17cyclictest0-21swapper/111:39:061
1532099201,17cyclictest0-21swapper/108:55:201
1530999207,2cyclictest21601-21needreboot07:20:200
1530999206,3cyclictest11469-21/usr/sbin/munin10:40:140
1530999205,3cyclictest24492-21ntp_states08:40:230
1530999204,3cyclictest87550irq/128-enp1s0-09:10:220
1530999204,10cyclictest0-21swapper/007:45:370
1530999202,6cyclictest0-21swapper/010:21:590
1530999202,6cyclictest0-21swapper/009:16:100
1530999201,8cyclictest0-21swapper/011:21:080
1530999201,16cyclictest16514-21strings07:10:180
1533099199,2cyclictest18412-21/usr/sbin/munin07:15:123
1533099195,5cyclictest0-21swapper/310:20:203
1533099192,9cyclictest18360-21memory09:40:183
1533099192,4cyclictest12186-21sshd08:14:573
1533099192,10cyclictest87650irq/129-enp1s0-08:24:453
1533099191,10cyclictest0-21swapper/311:16:443
15330991910,7cyclictest495-21jbd2/dm-0-808:45:253
1533099190,9cyclictest0-21swapper/309:51:303
1533099190,8cyclictest0-21swapper/309:35:583
1532599196,3cyclictest0-21swapper/212:32:412
1532599192,3cyclictest0-21swapper/211:57:062
1532099191,8cyclictest14702-21runrttasks09:31:081
15320991915,3cyclictest0-21swapper/111:11:131
15320991915,3cyclictest0-21swapper/107:34:431
1532099191,3cyclictest0-21swapper/108:39:201
1532099191,3cyclictest0-21swapper/108:20:031
15320991912,4cyclictest3283-21grep10:20:151
1532099191,17cyclictest0-21swapper/108:54:221
1532099191,16cyclictest0-21swapper/112:18:091
1532099191,16cyclictest0-21swapper/110:47:331
1532099191,16cyclictest0-21swapper/110:15:151
1532099191,16cyclictest0-21swapper/109:35:061
1532099191,16cyclictest0-21swapper/109:16:191
1532099191,16cyclictest0-21swapper/108:41:451
1532099191,16cyclictest0-21swapper/107:12:081
1532099190,5cyclictest0-21swapper/109:57:011
1532099190,4cyclictest0-21swapper/112:24:241
1532099190,17cyclictest0-21swapper/111:05:111
1532099190,17cyclictest0-21swapper/109:20:251
1532099190,17cyclictest0-21swapper/108:16:181
1530999196,3cyclictest10913-21netstat11:50:190
1530999195,3cyclictest0-21swapper/012:34:060
1530999191,7cyclictest0-21swapper/011:39:050
1530999191,7cyclictest0-21swapper/011:06:280
1530999191,6cyclictest20260-21/usr/sbin/munin11:00:200
1530999191,6cyclictest0-21swapper/010:46:560
1530999191,6cyclictest0-21swapper/009:46:570
1530999191,6cyclictest0-21swapper/009:46:570
1530999191,17cyclictest0-21swapper/012:28:270
1530999191,15cyclictest7303-21sshd11:43:050
1530999191,15cyclictest29150-21cat08:50:270
1530999190,8cyclictest0-21swapper/009:59:050
1530999190,8cyclictest0-21swapper/008:47:240
140062191,3sleep00-21swapper/011:59:340
1533099187,3cyclictest808-21snmpd08:57:493
1533099181,7cyclictest579-21systemd-journal07:50:013
15330991813,3cyclictest29930-21fschecks_time07:40:163
1533099181,14cyclictest839-21lldpd09:05:173
1532099184,3cyclictest0-21swapper/109:40:451
15320991815,2cyclictest0-21swapper/112:06:221
15320991815,2cyclictest0-21swapper/112:06:221
15320991814,3cyclictest0-21swapper/108:45:091
1532099181,3cyclictest0-21swapper/111:20:571
1532099181,16cyclictest11581-21kworker/u8:111:55:571
1532099181,16cyclictest0-21swapper/111:45:491
1532099181,16cyclictest0-21swapper/109:28:171
1532099181,16cyclictest0-21swapper/109:00:301
1532099181,16cyclictest0-21swapper/108:06:341
1532099181,16cyclictest0-21swapper/107:25:461
1532099181,14cyclictest5845-21diskstats11:40:141
1532099180,16cyclictest0-21swapper/111:30:031
1532099180,16cyclictest0-21swapper/110:30:241
1530999182,4cyclictest21222-21cstates12:15:130
1530999182,4cyclictest10008-21munin-run08:10:010
1530999182,3cyclictest29680-21/usr/sbin/munin12:35:210
1530999181,6cyclictest0-21swapper/012:12:110
1530999181,6cyclictest0-21swapper/010:55:170
1530999181,6cyclictest0-21swapper/010:55:170
1530999181,3cyclictest13050-21/usr/sbin/munin09:30:140
174702171,2sleep30-21swapper/307:14:563
1533099174,4cyclictest7416-21/usr/sbin/munin08:00:263
1533099172,3cyclictest87650irq/129-enp1s0-07:50:153
1533099171,9cyclictest0-21swapper/307:35:173
1533099171,9cyclictest0-21swapper/307:25:243
1533099171,6cyclictest25384-21latency_hist08:45:023
1533099171,5cyclictest19193-21/usr/sbin/munin08:30:133
15330991712,3cyclictest827-21runrttasks08:50:233
1533099171,11cyclictest87350irq/126-enp1s0-08:38:213
1530999172,14cyclictest0-21swapper/007:18:000
1530999171,3cyclictest25007-21/usr/sbin/munin07:30:130
15309991713,3cyclictest0-21swapper/007:58:140
15309991711,3cyclictest8134-21ntpq10:30:180
1530999171,13cyclictest12430-21/usr/sbin/munin08:15:160
15309991711,3cyclictest11189-21df_inode09:25:130
1533099165,3cyclictest87350irq/126-enp1s0-07:30:263
1533099161,7cyclictest12430-21/usr/sbin/munin08:15:133
1533099161,6cyclictest32207-21/usr/sbin/munin09:00:133
15309991610,3cyclictest23152-21sh07:25:140
15309991610,3cyclictest21532-21cat08:35:140
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional