You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-24 - 20:35

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack5slot3s.osadl.org (updated Wed Jun 24, 2026 12:44:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
88972236164,24sleep20-21swapper/207:08:422
88352225194,21sleep00-21swapper/007:07:510
86942203166,24sleep10-21swapper/107:06:041
87982202166,24sleep30-21swapper/307:07:263
1217321640,4sleep2909799cyclictest10:55:112
2731021270,2sleep3910299cyclictest10:15:153
9097991160,114cyclictest0-21swapper/207:40:482
9097991101,107cyclictest0-21swapper/208:04:352
9097997976,2cyclictest0-21swapper/210:29:512
81732580,1sleep30-21swapper/309:30:253
233542580,2sleep00-21swapper/007:40:130
281762560,1sleep10-21swapper/111:30:181
3952470,1sleep00-21swapper/009:15:140
135962450,1sleep213591-21interrupts08:30:162
908599411,33cyclictest8916-21kworker/0:107:14:450
908599381,28cyclictest31394-21kworker/0:008:04:210
909099377,23cyclictest31161-21kworker/u8:208:42:191
908599372,28cyclictest10294-21kworker/0:308:24:170
908599351,25cyclictest23450-21kworker/0:009:01:260
908599351,21cyclictest25017-21kworker/0:207:51:290
910299341,24cyclictest0-21swapper/309:39:193
910299341,16cyclictest0-21swapper/309:25:413
909099341,15cyclictest0-21swapper/110:32:111
908599332,25cyclictest7468-21kworker/0:212:01:390
36772330,4sleep232-21ksoftirqd/209:20:242
168872330,5sleep30-21swapper/311:05:163
910299321,9cyclictest0-21swapper/307:40:333
910299321,12cyclictest0-21swapper/308:52:413
909099321,14cyclictest0-21swapper/109:16:011
909099321,13cyclictest0-21swapper/108:34:251
909099321,12cyclictest0-21swapper/107:37:341
909099321,11cyclictest0-21swapper/107:15:231
910299310,11cyclictest0-21swapper/312:35:193
909099311,21cyclictest0-21swapper/111:10:051
909099311,15cyclictest27154-21chrt07:47:531
909099311,12cyclictest0-21swapper/112:30:411
909099311,11cyclictest25750-21munin-run09:00:011
908599311,19cyclictest8914-21kworker/0:211:10:180
908599311,14cyclictest25418-21munin-run12:40:010
909099301,12cyclictest0-21swapper/109:02:531
909099301,11cyclictest0-21swapper/109:40:181
909099301,10cyclictest15962-21users07:20:251
909099301,10cyclictest0-21swapper/108:15:191
908599301,21cyclictest31394-21kworker/0:008:07:570
910299291,10cyclictest0-21swapper/310:53:063
910299290,10cyclictest0-21swapper/309:08:333
909099291,15cyclictest0-21swapper/108:38:011
909099291,11cyclictest0-21swapper/108:51:331
909099291,10cyclictest0-21swapper/107:42:131
909099290,14cyclictest0-21swapper/109:46:491
909099290,12cyclictest0-21swapper/112:25:431
908599292,8cyclictest8914-21kworker/0:211:00:400
910299281,22cyclictest0-21swapper/309:47:433
909099288,6cyclictest31111-21cpuspeed_turbos10:25:121
909099281,2cyclictest0-21swapper/112:14:181
9090992812,2cyclictest0-21swapper/108:00:211
909099281,15cyclictest0-21swapper/110:14:571
909099281,12cyclictest0-21swapper/110:57:371
909099281,12cyclictest0-21swapper/108:21:051
909099281,11cyclictest0-21swapper/110:23:211
909099281,11cyclictest0-21swapper/109:59:531
909099281,11cyclictest0-21swapper/109:59:531
909099281,11cyclictest0-21swapper/109:20:061
909099281,10cyclictest0-21swapper/110:19:331
909099280,12cyclictest0-21swapper/112:03:421
908599288,2cyclictest27703-21kworker/0:309:07:060
908599287,10cyclictest0-21swapper/010:55:010
908599285,2cyclictest20515-21kworker/0:007:34:050
908599285,2cyclictest20515-21kworker/0:007:34:050
908599281,13cyclictest8173-21munin-run08:20:000
910299272,10cyclictest23775-21/usr/sbin/munin08:55:203
910299271,9cyclictest0-21swapper/312:28:533
910299271,9cyclictest0-21swapper/311:49:023
910299271,4cyclictest0-21swapper/312:30:083
910299271,23cyclictest0-21swapper/311:35:203
910299271,10cyclictest30160-21diskmemload10:45:183
909099271,9cyclictest0-21swapper/109:36:481
909099271,9cyclictest0-21swapper/108:25:211
909099271,8cyclictest0-21swapper/107:33:201
909099271,8cyclictest0-21swapper/107:33:201
909099271,13cyclictest0-21swapper/110:40:251
909099271,13cyclictest0-21swapper/110:36:131
909099271,12cyclictest0-21swapper/112:20:171
909099271,12cyclictest0-21swapper/111:46:331
909099271,12cyclictest0-21swapper/110:52:571
909099271,11cyclictest0-21swapper/111:05:531
909099271,10cyclictest0-21swapper/111:01:571
909099271,10cyclictest0-21swapper/109:25:081
909099270,11cyclictest0-21swapper/110:46:411
908599278,2cyclictest25477-21kworker/0:110:13:530
908599272,10cyclictest22253-21kworker/0:311:20:340
908599271,6cyclictest18971-21kworker/0:208:53:220
910299261,7cyclictest0-21swapper/308:04:213
910299261,14cyclictest16589-40dnf12:19:433
910299261,14cyclictest16589-40dnf12:19:433
910299261,10cyclictest0-21swapper/308:05:303
910299260,9cyclictest0-21swapper/308:24:183
909099261,9cyclictest0-21swapper/110:02:091
909099261,17cyclictest24811-21idleruntime-cro11:25:011
909099261,12cyclictest0-21swapper/112:16:221
909099261,12cyclictest0-21swapper/112:16:211
909099261,12cyclictest0-21swapper/111:57:541
909099261,12cyclictest0-21swapper/109:52:371
909099261,11cyclictest0-21swapper/110:07:131
909099261,10cyclictest0-21swapper/111:28:011
909099261,10cyclictest0-21swapper/109:32:221
909099261,10cyclictest0-21swapper/107:57:311
908599266,2cyclictest18971-21kworker/0:208:49:540
908599264,2cyclictest10263-21kworker/0:008:25:450
908599262,2cyclictest87650irq/129-enp1s0-07:17:490
908599261,12cyclictest30781-21munin-run10:25:010
910299255,3cyclictest0-21swapper/308:44:213
910299253,14cyclictest87350irq/126-enp1s0-11:10:203
910299251,9cyclictest0-21swapper/311:41:513
910299251,8cyclictest22535-40dnf11:19:243
910299251,13cyclictest0-21swapper/310:35:393
910299251,11cyclictest0-21swapper/312:14:213
910299250,9cyclictest0-21swapper/312:01:393
910299250,16cyclictest0-21swapper/310:40:263
910299250,11cyclictest0-21swapper/312:05:573
910299250,10cyclictest0-21swapper/311:28:433
909099251,14cyclictest20629-21/usr/sbin/munin11:15:251
909099251,11cyclictest0-21swapper/112:05:541
909099251,11cyclictest0-21swapper/111:53:501
909099251,10cyclictest0-21swapper/109:11:171
908599251,14cyclictest31311-21kworker/0:111:52:250
908599250,9cyclictest7865-21kworker/0:109:45:550
910299249,3cyclictest28288-21ntp_states11:30:203
910299248,3cyclictest0-21swapper/307:45:453
910299247,3cyclictest0-21swapper/307:58:193
910299246,3cyclictest0-21swapper/310:34:403
910299244,3cyclictest0-21swapper/308:30:323
910299241,9cyclictest0-21swapper/309:56:103
910299241,9cyclictest0-21swapper/309:56:103
910299241,7cyclictest2143-21chrt09:19:333
910299241,6cyclictest5636-21sleep11:55:003
910299241,20cyclictest0-21swapper/310:14:073
910299241,15cyclictest0-21swapper/311:57:353
910299241,15cyclictest0-21swapper/310:26:193
910299241,12cyclictest19384-21/usr/sbin/munin08:45:163
910299241,12cyclictest0-21swapper/310:03:273
910299240,9cyclictest0-21swapper/310:57:583
910299240,10cyclictest0-21swapper/310:24:023
910299240,10cyclictest0-21swapper/308:12:143
909099241,7cyclictest0-21swapper/111:40:461
909099241,10cyclictest0-21swapper/108:11:091
909099241,10cyclictest0-21swapper/107:53:131
908599244,2cyclictest8914-21kworker/0:210:46:510
908599244,10cyclictest0-21swapper/010:01:430
908599242,2cyclictest89250irq/135-enp2s0-11:37:240
9085992420,3cyclictest0-21swapper/007:35:240
908599241,3cyclictest87550irq/128-enp1s0-08:59:580
908599240,15cyclictest26818-21kworker/0:111:31:000
910299237,5cyclictest0-21swapper/310:06:263
910299232,10cyclictest10602-21/usr/sbin/munin08:25:223
910299231,11cyclictest9286-21/usr/sbin/munin07:10:193
910299231,11cyclictest25971-21/usr/sbin/munin09:00:153
910299231,11cyclictest0-21swapper/311:24:433
910299231,11cyclictest0-21swapper/309:13:423
910299231,10cyclictest808-21snmpd07:31:403
910299231,10cyclictest808-21snmpd07:31:403
910299230,21cyclictest0-21swapper/311:00:213
909799230,10cyclictest0-21swapper/211:24:352
909099231,14cyclictest0-21swapper/107:11:131
909099231,11cyclictest0-21swapper/111:35:211
909099231,11cyclictest0-21swapper/109:05:091
908599239,2cyclictest31311-21kworker/0:111:45:460
908599232,9cyclictest2087-21kworker/0:209:29:190
908599232,11cyclictest28295-21kworker/0:310:19:310
908599232,10cyclictest8914-21kworker/0:210:58:310
908599232,10cyclictest7865-21kworker/0:109:32:270
908599231,7cyclictest0-21swapper/011:29:240
908599231,2cyclictest7865-21kworker/0:109:40:270
9085992311,3cyclictest3453-21fschecks_count10:35:140
908599230,3cyclictest27259-21kworker/0:107:57:410
910299228,3cyclictest11211-21/usr/sbin/munin09:40:193
910299225,3cyclictest0-21swapper/307:51:333
910299224,3cyclictest0-21swapper/307:29:333
910299223,7cyclictest0-21swapper/307:16:303
910299221,8cyclictest0-21swapper/309:22:273
910299221,14cyclictest0-21swapper/312:21:413
910299221,10cyclictest0-21swapper/309:52:283
910299220,20cyclictest0-21swapper/307:20:133
909799228,3cyclictest0-21swapper/208:43:352
909099221,12cyclictest0-21swapper/107:29:221
909099221,10cyclictest0-21swapper/108:47:571
908599224,5cyclictest0-21swapper/010:31:430
908599224,4cyclictest18800-21uname09:55:190
908599224,4cyclictest18800-21uname09:55:190
908599223,2cyclictest27703-21kworker/0:309:12:100
908599222,6cyclictest32244-21kworker/0:109:21:350
908599222,5cyclictest16907-21kworker/0:008:42:160
908599222,2cyclictest22248-21kworker/0:007:21:420
908599222,11cyclictest7468-21kworker/0:212:23:510
908599221,9cyclictest10294-21kworker/0:308:32:410
908599221,11cyclictest7468-21kworker/0:212:06:410
157502220,5sleep011-21rcuc/009:50:120
910299211,11cyclictest0-21swapper/308:37:573
910299211,10cyclictest0-21swapper/308:19:303
909799216,6cyclictest5891-21kworker/u8:007:45:122
909099211,11cyclictest0-21swapper/108:06:571
909099210,15cyclictest0-21swapper/112:39:141
908599212,10cyclictest22361-21cut12:30:220
908599212,10cyclictest17293-21ntp_states11:05:210
908599211,2cyclictest89050irq/133-enp2s0-07:28:270
908599211,11cyclictest7468-21kworker/0:212:25:270
908599211,11cyclictest12456-21/usr/sbin/munin12:10:140
908599211,10cyclictest19679-21kworker/0:210:09:350
908599210,2cyclictest87550irq/128-enp1s0-10:28:130
908599210,13cyclictest31394-21kworker/0:008:10:170
910299201,7cyclictest21235-21expr07:35:133
909799204,7cyclictest30-21rcuc/212:00:272
9097992013,4cyclictest18245-1kworker/u9:211:19:242
908599209,3cyclictest0-21swapper/008:38:430
908599208,2cyclictest20515-21kworker/0:007:45:470
908599206,6cyclictest31311-21kworker/0:111:41:420
908599204,3cyclictest31311-21kworker/0:111:56:330
908599204,2cyclictest14222-21kworker/0:012:15:550
908599204,2cyclictest14222-21kworker/0:012:15:550
908599200,10cyclictest7217-21perf10:45:000
909799193,6cyclictest30-21rcuc/212:15:142
909799193,6cyclictest30-21rcuc/212:15:142
909799192,4cyclictest7517-21kworker/2:108:53:582
909799192,14cyclictest12097-21grep08:25:272
909799191,8cyclictest958-21cpuspeed_turbos10:30:132
909799191,7cyclictest0-21swapper/211:45:382
909799191,6cyclictest0-21swapper/211:53:012
909799191,5cyclictest0-21swapper/212:37:232
909799191,5cyclictest0-21swapper/211:44:482
9097991914,3cyclictest23211-21idleruntime-cro12:35:022
9097991912,4cyclictest19317-21grep09:55:262
9097991912,4cyclictest19317-21grep09:55:262
909799191,12cyclictest0-21swapper/208:45:092
909799191,11cyclictest0-21swapper/212:20:272
909799191,10cyclictest0-21swapper/210:35:102
908599195,3cyclictest87550irq/128-enp1s0-11:18:440
909799185,3cyclictest23789-21head10:05:282
909799184,6cyclictest18814-21fschecks_count11:10:142
909799183,6cyclictest0-21swapper/212:00:012
909799183,2cyclictest29990-21runrttasks09:08:042
909799182,3cyclictest0-21swapper/210:48:072
909799182,15cyclictest0-21swapper/208:59:512
909799182,15cyclictest0-21swapper/207:11:102
909799181,8cyclictest6597-21sleep09:30:002
909799181,8cyclictest14763-21if_enp1s011:00:172
909799181,7cyclictest808-21snmpd09:50:132
909799181,7cyclictest0-21swapper/210:41:372
909799181,6cyclictest21025-21perf12:30:022
909799181,5cyclictest0-21swapper/210:17:202
909799180,11cyclictest0-21swapper/210:10:132
166322180,2sleep20-21swapper/208:35:592
909799175,3cyclictest20705-21sshd07:34:432
909799175,3cyclictest20705-21sshd07:34:432
909799174,4cyclictest2780-21cat08:05:202
909799173,5cyclictest0-21swapper/211:06:502
909799171,7cyclictest7655-21grep09:30:182
909799171,7cyclictest0-21swapper/211:33:142
909799171,7cyclictest0-21swapper/209:38:212
909799171,6cyclictest7487-21switchtime08:15:242
909799171,6cyclictest14425-21latency_hist12:15:012
909799171,6cyclictest11599-21crond10:55:002
909799171,6cyclictest0-21swapper/211:35:182
909799171,4cyclictest18245-1kworker/u9:207:17:402
909799171,3cyclictest0-21swapper/212:05:192
909799171,3cyclictest0-21swapper/211:25:082
9097991713,3cyclictest0-21swapper/207:50:042
9097991712,3cyclictest9476-21ntpq08:20:212
9097991712,3cyclictest4913-21memory08:10:182
909799171,14cyclictest0-21swapper/207:35:252
909799171,11cyclictest0-21swapper/209:40:202
908599171,5cyclictest0-21swapper/009:36:520
909799165,3cyclictest31462-21taskset07:56:462
909799161,7cyclictest0-21swapper/210:01:562
909799161,6cyclictest808-21snmpd09:47:082
909799161,6cyclictest0-21swapper/209:15:232
909799161,5cyclictest0-21swapper/209:10:042
9097991611,3cyclictest30146-21timerandwakeup10:20:242
909799160,14cyclictest0-21swapper/209:00:362
909799160,14cyclictest0-21swapper/207:25:032
909799151,7cyclictest0-21swapper/207:20:212
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional