You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-20 - 15:36

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackaslot8s.osadl.org (updated Sat Jun 20, 2026 12:45:17)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
27054221896,97sleep30-21swapper/307:09:023
27054221896,97sleep30-21swapper/307:09:013
26892221496,94sleep00-21swapper/007:06:540
26892221496,94sleep00-21swapper/007:06:530
27089216496,61sleep10-21swapper/107:09:281
27089216496,61sleep10-21swapper/107:09:271
2898121510,1sleep20-21swapper/210:56:452
269562142104,31sleep20-21swapper/207:07:452
269562142104,31sleep20-21swapper/207:07:442
267222870,2sleep30-21swapper/312:31:163
100472730,3sleep310045-21ssh11:55:163
327032650,2sleep20-21swapper/212:35:332
327032650,2sleep20-21swapper/212:35:332
151042560,3sleep30-21swapper/311:10:213
34252540,2sleep10-21swapper/110:38:011
256812540,1sleep00-21swapper/011:42:410
102032530,1sleep210207-21cpuspeed_turbos09:55:132
170162490,1sleep10-21swapper/109:11:091
113802490,1sleep20-21swapper/211:31:482
279112480,1sleep10-21swapper/112:08:351
221512460,2sleep00-21swapper/008:10:240
27302450,1sleep00-21swapper/010:13:410
32432440,2sleep10-21swapper/111:50:171
12742440,1sleep20-21swapper/207:25:002
62092430,2sleep20-21swapper/211:52:142
45882430,2sleep00-21swapper/012:39:530
45882430,2sleep00-21swapper/012:39:520
158932430,1sleep30-21swapper/309:10:233
141282420,1sleep20-21swapper/211:10:132
3432370,2sleep00-21swapper/009:47:240
308692370,2sleep20-21swapper/211:22:172
320962330,2chrt0-21swapper/110:10:361
150102310,2chrt0-21swapper/209:34:462
150102310,2chrt0-21swapper/209:34:452
99672300,3sleep334-21ksoftirqd/309:30:223
99672300,3sleep334-21ksoftirqd/309:30:223
276362300,2sleep20-21swapper/208:25:162
2740799300,27cyclictest10166-21grep09:00:133
27405993023,4cyclictest19482-21memory08:05:181
2740599291,15cyclictest0-21swapper/107:55:231
2740599291,15cyclictest0-21swapper/107:55:221
289722280,3sleep334-21ksoftirqd/311:45:143
2740599281,25cyclictest0-21swapper/108:22:511
2740499281,25cyclictest0-21swapper/008:42:150
84422270,3sleep00-21swapper/009:53:460
2740799271,24cyclictest0-21swapper/308:14:413
2740799271,24cyclictest0-21swapper/308:07:563
27406992722,3cyclictest5184-21users08:45:262
27406992722,3cyclictest5184-21users08:45:262
2740699271,24cyclictest0-21swapper/208:16:042
2740699271,24cyclictest0-21swapper/207:57:532
2740699271,24cyclictest0-21swapper/207:57:532
2740599271,25cyclictest0-21swapper/107:13:341
2740599271,24cyclictest0-21swapper/108:52:261
2740599271,24cyclictest0-21swapper/108:40:111
2740599271,24cyclictest0-21swapper/108:35:491
2740599271,24cyclictest0-21swapper/108:15:481
2740499271,2cyclictest0-21swapper/008:02:350
2740499271,25cyclictest0-21swapper/008:05:190
2740499271,24cyclictest0-21swapper/009:00:170
2740499271,24cyclictest0-21swapper/008:55:020
2740499271,24cyclictest0-21swapper/008:52:220
2740499271,24cyclictest0-21swapper/008:25:300
2740499271,24cyclictest0-21swapper/008:22:280
2740499271,24cyclictest0-21swapper/008:15:570
2740499271,24cyclictest0-21swapper/007:58:570
2740499271,24cyclictest0-21swapper/007:58:560
113812270,2sleep20-21swapper/212:20:162
294732260,2sleep00-21swapper/012:34:030
27407992622,2cyclictest0-21swapper/308:42:153
2740799261,2cyclictest0-21swapper/307:13:333
2740799261,24cyclictest0-21swapper/308:21:523
2740799261,23cyclictest0-21swapper/308:27:013
2740799261,23cyclictest0-21swapper/308:02:043
2740799260,24cyclictest0-21swapper/307:57:403
2740799260,24cyclictest0-21swapper/307:57:393
2740699261,24cyclictest0-21swapper/208:39:412
2740699261,24cyclictest0-21swapper/207:14:362
2740699261,23cyclictest0-21swapper/209:01:022
2740699261,23cyclictest0-21swapper/208:51:522
2740699261,23cyclictest0-21swapper/208:21:532
2740699261,23cyclictest0-21swapper/208:05:112
2740699261,23cyclictest0-21swapper/207:15:092
2740599263,19cyclictest27822-21grep11:20:161
2740599261,24cyclictest0-21swapper/108:03:011
2740599261,1cyclictest0-21swapper/108:26:061
2740599261,1cyclictest0-21swapper/107:15:041
2740599260,24cyclictest0-21swapper/109:00:171
2740599260,24cyclictest0-21swapper/108:10:161
2740499261,24cyclictest0-21swapper/008:37:390
2740499261,24cyclictest0-21swapper/007:13:420
2740499261,1cyclictest0-21swapper/008:45:200
2740499261,1cyclictest0-21swapper/008:45:190
200332260,2sleep20-21swapper/209:14:102
129122260,2sleep30-21swapper/310:20:253
27407992523,1cyclictest0-21swapper/308:15:473
2740799251,23cyclictest0-21swapper/308:59:503
2740799251,23cyclictest0-21swapper/308:50:553
2740799251,23cyclictest0-21swapper/308:37:183
2740799250,23cyclictest0-21swapper/308:45:203
2740799250,23cyclictest0-21swapper/308:45:193
27406992523,1cyclictest0-21swapper/208:04:562
2740699251,23cyclictest0-21swapper/208:40:032
2740699251,23cyclictest0-21swapper/208:10:202
2740699250,23cyclictest0-21swapper/209:06:152
2740599257,6cyclictest22-21ksoftirqd/112:01:071
27404992515,7cyclictest19599-21df10:50:150
106992250,2sleep10-21swapper/110:43:361
31732240,2sleep30-21swapper/310:14:073
277322240,10sleep012-21ksoftirqd/009:44:030
27407992422,1cyclictest0-21swapper/308:30:033
27406992415,5cyclictest14143-21diskmemload10:05:242
2740699241,1cyclictest0-21swapper/208:56:092
2740599243,7cyclictest0-21swapper/109:25:241
27405992422,1cyclictest0-21swapper/108:59:511
27405992421,2cyclictest0-21swapper/108:30:031
27405992420,2cyclictest0-21swapper/109:05:571
27404992414,6cyclictest52850irq/140-enp3s0-12:25:110
227572240,2sleep20-21swapper/212:29:022
163022240,5sleep0131rcuc/009:35:160
78682230,2sleep20-21swapper/209:53:112
291832230,1sleep10-21swapper/111:45:151
27405992320,2cyclictest0-21swapper/108:45:221
27405992320,2cyclictest0-21swapper/108:45:211
2740599231,9cyclictest3975-21/usr/sbin/munin12:15:231
2740599231,5cyclictest0-21swapper/112:20:141
27404992321,1cyclictest0-21swapper/008:30:030
27404992314,6cyclictest6143-21awk10:15:260
213332230,2sleep10-21swapper/109:39:221
202692230,2sleep10-21swapper/110:26:061
198652230,1sleep30-21swapper/312:26:073
156282230,2sleep10-21swapper/111:10:281
149052230,2sleep20-21swapper/210:22:262
42632220,2sleep10-21swapper/112:39:331
42632220,2sleep10-21swapper/112:39:331
277642220,2sleep10-21swapper/112:32:181
27407992220,1cyclictest0-21swapper/309:05:573
27407992211,8cyclictest29734-21apt-get12:10:123
27407992210,8cyclictest31529-21apt-get10:35:133
27406992212,7cyclictest30407-21tr09:45:212
27406992212,6cyclictest52950irq/141-enp3s0-11:25:122
27405992213,7cyclictest8659-21diskstats11:30:151
27405992213,6cyclictest10007-21netstat09:30:221
27405992213,6cyclictest10007-21netstat09:30:221
27404992216,4cyclictest9903-21sh07:45:000
27404992211,8cyclictest5395-21cron10:40:010
214242220,2sleep30-21swapper/310:03:213
129402220,2sleep10-21swapper/109:56:321
85562210,10sleep334-21ksoftirqd/311:54:373
2740799213,2cyclictest141rcu_preempt12:02:143
2740799212,2cyclictest141rcu_preempt11:26:283
2740799211,6cyclictest26318-21df10:55:143
27407992114,5cyclictest2601-21mailstats07:25:243
27407992113,6cyclictest7013-21sensors_temp07:35:243
27407992113,6cyclictest6198-21sed10:15:263
27407992113,6cyclictest23987-21sort10:05:163
2740799211,10cyclictest113802ssh12:20:163
27406992111,7cyclictest20568-21tr10:50:232
2740699211,14cyclictest9402-21sh11:55:132
2740599219,8cyclictest22291-21tail11:40:171
2740599214,4cyclictest14143-21diskmemload09:49:291
2740599213,4cyclictest14143-21diskmemload09:51:391
2740599213,4cyclictest14143-21diskmemload09:16:461
2740599212,7cyclictest22-21ksoftirqd/110:58:531
27405992116,3cyclictest23076-21sh10:04:591
2740499218,11cyclictest15258-21turbostat09:35:000
2740499218,11cyclictest15258-21turbostat09:34:590
27404992114,4cyclictest11164-21/usr/sbin/munin10:20:190
27404992112,6cyclictest24142-21awk12:05:220
27404992111,8cyclictest17168-21rm11:35:550
2740499211,15cyclictest12489-21kworker/u8:211:05:040
2740799204,3cyclictest14143-21diskmemload09:48:293
2740799203,3cyclictest14143-21diskmemload10:34:013
2740799203,3cyclictest14143-21diskmemload09:43:193
27407992012,6cyclictest4725-21smartctl09:50:243
27407992012,6cyclictest31636-21kernelversion12:35:193
27407992012,6cyclictest31636-21kernelversion12:35:193
27407992011,7cyclictest560-21dbus-daemon09:29:593
27407992010,8cyclictest21705-21rm10:51:103
27407992010,6cyclictest4019-21ssh12:15:113
27406992015,3cyclictest28715-21idleruntime-cro09:45:002
27406992012,6cyclictest8169-21ntpq11:05:192
27406992012,6cyclictest30573-21date12:10:182
27406992012,5cyclictest52950irq/141-enp3s0-11:35:142
2740599208,8cyclictest8017-21apt-get07:40:131
2740599204,3cyclictest141rcu_preempt09:40:271
2740599204,3cyclictest14143-21diskmemload10:22:041
2740599204,13cyclictest7767-21sed11:05:161
2740599203,3cyclictest14143-21diskmemload12:25:301
2740599203,3cyclictest14143-21diskmemload10:07:511
2740599202,4cyclictest14143-21diskmemload11:56:101
2740599201,8cyclictest5492-21netstat10:15:201
27405992012,6cyclictest28667-21cat09:20:161
27405992012,6cyclictest24955-21df10:30:141
2740499208,3cyclictest0-21swapper/012:00:010
2740499201,6cyclictest14595-21ssh09:58:120
27404992013,4cyclictest7664-21sh11:29:500
27404992013,4cyclictest17682-21sh10:25:010
27404992012,6cyclictest278-21systemd-journal07:30:010
27404992012,6cyclictest10755-21cat07:45:170
27404992012,5cyclictest14143-21diskmemload11:24:500
27404992011,6cyclictest12437-21seq10:45:070
2740499200,11cyclictest141rcu_preempt10:56:310
211952200,2sleep30-21swapper/309:15:083
2740799194,13cyclictest22002-21grep09:40:003
2740799193,3cyclictest14143-21diskmemload12:06:213
2740799191,5cyclictest7151-21apt-get11:05:123
27407991914,3cyclictest16292-21sh10:47:273
27407991913,4cyclictest8401-21sh10:41:163
27407991913,4cyclictest30871-21sh11:22:163
27407991913,4cyclictest278-21systemd-journal07:54:553
27407991912,5cyclictest28942-21grep09:20:193
27407991911,6cyclictest15510-21fschecks_time11:35:163
27407991911,5cyclictest30111-21sed07:15:163
27407991910,6cyclictest10894-21latency09:55:173
2740699192,13cyclictest24616-21ssh10:30:122
2740699191,8cyclictest18455-21grep10:00:262
2740699191,3cyclictest18519-21cut10:25:162
27406991912,5cyclictest31389-21ps10:10:232
27406991912,5cyclictest28-21ksoftirqd/211:50:002
27406991912,5cyclictest28-21ksoftirqd/211:05:012
27406991912,5cyclictest28-21ksoftirqd/211:05:002
27406991912,5cyclictest24391-21sensors_temp12:05:252
27406991911,6cyclictest560-21dbus-daemon07:40:002
2740699191,15cyclictest25655-21timerwakeupswit12:30:242
2740699191,13cyclictest31380-21apt-get10:35:042
27406991910,7cyclictest560-21dbus-daemon11:20:012
2740599198,9cyclictest0-21swapper/110:50:151
2740599193,3cyclictest14143-21diskmemload11:01:441
2740599193,3cyclictest14143-21diskmemload11:01:441
2740599191,4cyclictest13275-21irqstats10:45:171
27405991914,3cyclictest30515-21grep12:10:171
27405991911,6cyclictest5585-21sed07:35:011
2740599191,14cyclictest14143-21diskmemload11:25:121
27405991910,7cyclictest1223-21cron07:25:011
2740499196,3cyclictest89681sh11:30:170
27404991914,3cyclictest31496-21sh09:22:140
27404991914,3cyclictest26112-21sh09:18:330
27404991913,4cyclictest23952-21tune2fs10:05:150
27404991911,6cyclictest4804-21sed07:30:230
27404991911,6cyclictest4385-21grep11:50:270
27404991911,6cyclictest31631-21seq12:10:350
27404991911,6cyclictest21453-21cat11:15:180
27404991911,6cyclictest14559-21seq12:22:310
27404991911,6cyclictest12372-21grep09:05:140
27404991911,5cyclictest32224-21grep07:20:150
2740499191,15cyclictest0-21swapper/012:03:050
27404991910,7cyclictest65511rm11:05:000
27404991910,7cyclictest65511rm11:05:000
251622190,2chrt0-21swapper/209:17:372
27407991813,3cyclictest9333-21ntp_offset11:30:213
27407991813,3cyclictest835-21uptime07:20:263
27407991813,3cyclictest2277-21sh11:00:443
27407991813,3cyclictest2277-21sh11:00:433
27407991813,3cyclictest22651-21sh11:15:483
27407991813,3cyclictest21661-21sh11:40:113
27407991812,4cyclictest20603-21sh10:26:263
2740799181,12cyclictest3615-21apt-get07:30:133
2740699181,6cyclictest0-21swapper/212:00:122
2740699181,4cyclictest0-21swapper/209:25:102
27406991813,3cyclictest30902-21timerandwakeup08:30:232
27406991813,3cyclictest16443-21sh09:35:182
27406991812,4cyclictest636-21runrttasks09:20:192
27406991812,4cyclictest5819-21sensors_temp10:15:232
27406991811,5cyclictest5897-21awk10:40:112
27406991811,4cyclictest12483-21/usr/sbin/munin10:45:142
27406991810,6cyclictest8813-21grep07:40:192
27406991810,6cyclictest5339-21seq07:33:042
27406991810,6cyclictest5110-21pluginstate12:15:202
2740599189,6cyclictest2910-21grep07:25:251
2740599182,14cyclictest14176-21munin-run07:54:591
27405991812,4cyclictest11103-21open_inodes07:45:221
27405991811,5cyclictest6485-21cat07:35:191
27405991811,5cyclictest53050irq/142-enp3s0-11:35:161
2740599181,11cyclictest636-21runrttasks11:15:111
2740499182,11cyclictest5820-21apt-get10:40:110
2740499181,8cyclictest16561-21rm09:10:420
27404991814,3cyclictest14349-21fschecks_time11:10:150
27404991812,4cyclictest6566-21sh12:16:150
27404991812,4cyclictest636-21runrttasks10:30:550
27404991812,4cyclictest5637-21sh09:27:040
27404991811,5cyclictest28348-21cat11:45:090
27404991811,5cyclictest13598-21ls07:50:240
27404991810,5cyclictest16885-21cat10:00:130
27407991710,5cyclictest10521-21cut07:45:153
27406991712,3cyclictest21682-21/usr/sbin/munin11:40:232
27406991712,3cyclictest1453-21/usr/sbin/munin07:25:172
27406991710,5cyclictest12490-21expr07:50:132
2740499179,6cyclictest6434-21kernelversion07:35:200
2740799168,6cyclictest9260-21awk07:40:253
27406991611,3cyclictest10128-21/usr/sbin/munin07:45:152
2740499159,4cyclictest29834-21expr07:15:140
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional