You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-30 - 16:51

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Tue Dec 30, 2025 12:43:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1220729581,9sleep00-21swapper/007:08:150
1221029467,9sleep30-21swapper/307:08:183
1204028667,9sleep20-21swapper/207:06:042
1228728063,13sleep10-21swapper/107:09:161
12426992726,1cyclictest0-21swapper/011:13:420
12437992623,2cyclictest39-21ksoftirqd/311:55:003
1243099231,19cyclictest0-21swapper/111:35:131
12435991918,1cyclictest0-21swapper/209:25:142
12430991918,1cyclictest0-21swapper/108:46:471
1242699192,16cyclictest7090-21apt-get10:20:130
12437991814,3cyclictest4133-21uniq09:05:173
1243799180,12cyclictest0-21swapper/311:40:143
12437991513,1cyclictest381rcuc/310:15:003
12437991512,2cyclictest9183-21cron10:25:003
1242699159,2cyclictest475-21in:imuxsock08:28:350
1243799144,8cyclictest16453-21apt08:25:003
12437991410,2cyclictest4697-21tr10:15:133
1243099141,4cyclictest0-21swapper/111:40:131
12430991411,2cyclictest25-21ksoftirqd/110:30:271
1242699141,4cyclictest0-21swapper/009:40:140
1243799139,3cyclictest1206-21cat10:05:283
1243799134,5cyclictest20264-21apt-get09:40:143
12435991310,2cyclictest23405-21grep09:45:172
12435991310,2cyclictest21352-21cat07:25:262
1243599130,12cyclictest7709-21grep10:20:172
12430991310,2cyclictest20027-21wc10:45:201
1242699139,3cyclictest29352-21cut08:50:190
12426991310,2cyclictest6578-21cron10:20:000
12426991310,2cyclictest24140-21wc09:45:250
12426991310,2cyclictest14318-21tr10:35:140
1243799129,2cyclictest8056-21cat11:30:013
1243799129,2cyclictest5276-21grep12:30:173
1243799129,2cyclictest32596-21grep12:20:163
1243799129,2cyclictest30945-21cat11:10:013
1243799129,2cyclictest30867-21cat10:00:203
1243799129,2cyclictest28284-21cat07:40:193
1243799129,2cyclictest25609-21expr12:05:163
1243799129,2cyclictest24035-21grep12:00:233
1243799129,2cyclictest18247-21cat08:25:263
1243799129,2cyclictest1359-21tr11:15:173
1243799129,2cyclictest12724-21sh11:40:003
1243799129,2cyclictest11393-21grep09:20:193
1243799129,2cyclictest10162-21taskset11:32:463
12437991210,1cyclictest20779-21grep08:30:263
12437991210,1cyclictest14216-21apt-get10:35:143
12437991210,1cyclictest13958-21basename10:34:593
12437991210,1cyclictest1024-21cat07:50:253
1243799120,11cyclictest9571-21grep08:10:153
1243799120,11cyclictest8995-21head09:15:173
1243599129,2cyclictest810-21cron11:15:002
1243599129,2cyclictest30876-21dump-pmu-power11:10:002
1243599129,2cyclictest28099-21df12:10:172
1243599129,2cyclictest22335-21tr12:00:002
1243599129,2cyclictest17414-21grep08:25:172
1243599129,2cyclictest16669-21grep11:45:222
1243599129,2cyclictest16131-21cat07:15:192
1243599129,2cyclictest15038-21head10:35:172
1243599122,4cyclictest31755-21sshd12:16:342
12435991210,1cyclictest615-21grep12:20:182
12435991210,1cyclictest29382-21head08:50:192
12435991210,1cyclictest21190-21ls08:35:002
12435991210,1cyclictest14832-21idleruntime-cro07:15:002
12435991210,1cyclictest0-21swapper/211:00:172
1243599120,3cyclictest0-21swapper/207:50:132
1243099129,2cyclictest30065-21grep12:15:151
1243099129,2cyclictest20991-21chrt08:32:371
1243099129,2cyclictest12644-21cut08:15:181
1243099128,3cyclictest1005-21cut12:20:221
1243099128,2cyclictest25-21ksoftirqd/111:50:131
1243099128,1cyclictest16398-21basename10:40:001
1243099126,2cyclictest225-21systemd-journal10:20:001
1243099126,1cyclictest225-21systemd-journal11:58:341
1243099121,5cyclictest0-21swapper/109:35:131
12430991210,1cyclictest9197-21turbostat.cron10:25:001
12430991210,1cyclictest20103-21tr07:25:141
1243099120,11cyclictest21941-21grep09:40:271
1242699129,2cyclictest9513-21tr08:10:150
1242699129,2cyclictest8124-21wc09:15:010
1242699129,2cyclictest32028-21wc10:05:000
1242699129,2cyclictest30672-21grep12:15:170
1242699129,2cyclictest2879-21grep07:55:160
1242699129,2cyclictest25116-21tr07:35:150
1242699129,2cyclictest23543-21grep12:00:180
1242699129,2cyclictest20597-21grep07:25:170
1242699129,2cyclictest15394-21wc08:20:200
12426991210,1cyclictest12873-21ls10:30:210
1243799119,1cyclictest6481-21taskset08:03:033
1243799119,1cyclictest4230-21head11:20:173
1243799119,1cyclictest4048-21seq12:25:483
1243799119,1cyclictest31742sleep309:04:013
1243799119,1cyclictest29473-21cron10:00:003
1243799119,1cyclictest26578-21expr08:45:163
1243799119,1cyclictest25741-21chrt08:42:253
1243799119,1cyclictest25116-21awk09:50:143
1243799119,1cyclictest20741-21df11:55:163
1243799119,1cyclictest20174-21cat07:25:143
1243799119,1cyclictest18768-21sh10:45:003
1243799119,1cyclictest15894-21cat11:45:153
1243799119,1cyclictest15592-21hostname09:30:163
1243799119,1cyclictest12624-21chrt07:10:053
1243799118,2cyclictest7102-21egrep08:05:183
1243799118,2cyclictest3273-21ntpq07:55:203
1243799118,2cyclictest30712-21uname07:45:213
1243799118,2cyclictest29689-21cat12:15:003
1243799118,2cyclictest29317-21head08:50:193
1243799118,2cyclictest23596-21tr08:40:003
1243799118,2cyclictest23038-21cat09:45:143
1243799118,2cyclictest15076-21cat07:15:123
1243799118,2cyclictest12131-21tr08:15:163
1243799114,6cyclictest25825-21sshd10:56:343
1243799113,3cyclictest24792-21apt-get07:35:033
1243799111,8cyclictest0-21swapper/309:35:133
1243599119,1cyclictest9900-21grep09:15:272
1243599119,1cyclictest9147-21basename12:40:002
1243599119,1cyclictest28370-21ntpq07:40:212
1243599119,1cyclictest26624-21expr08:45:162
1243599119,1cyclictest25935-21grep07:35:212
1243599119,1cyclictest18715-21sed07:20:202
1243599118,2cyclictest7513-21grep08:05:202
1243599118,2cyclictest23568-21ls08:40:002
1243599118,2cyclictest23307-21grep10:50:272
1243599118,2cyclictest18074-21cat09:35:142
1243599116,2cyclictest225-21systemd-journal09:00:002
1243599116,2cyclictest225-21systemd-journal09:00:002
1243599112,4cyclictest6558-21sshd12:32:342
1243599111,4cyclictest0-21swapper/212:05:152
1243099119,1cyclictest5962sleep108:59:341
1243099119,1cyclictest5962sleep108:59:341
1243099119,1cyclictest4767-21tr08:00:161
1243099119,1cyclictest32375-21tr11:10:211
1243099119,1cyclictest314-21cut10:05:191
1243099119,1cyclictest26955-21chrt12:05:531
1243099119,1cyclictest25755-21taskset10:55:381
1243099119,1cyclictest173762sleep111:47:501
1243099119,1cyclictest161352sleep108:21:111
1243099119,1cyclictest15174-21tr07:15:141
1243099118,2cyclictest7738-21head12:35:161
1243099118,2cyclictest5168-21sed11:20:271
1243099118,2cyclictest443-21cut07:50:191
1243099118,2cyclictest31712-21turbostat07:50:001
1243099118,2cyclictest30320-21grep10:00:161
1243099118,2cyclictest29315-21grep08:50:191
1243099118,2cyclictest28097-21head09:55:171
1243099118,2cyclictest26511-21cut11:00:141
1243099118,2cyclictest23307-21cat07:30:211
1243099118,2cyclictest16916-21tr10:40:171
1243099118,2cyclictest14765-21sed09:25:271
1243099118,2cyclictest14205-21wc07:10:261
1243099118,2cyclictest10503-21cat10:25:221
1243099118,2cyclictest10402-21sleep09:20:001
1243099117,3cyclictest28651-21cat07:40:251
1243099117,3cyclictest18810-21cron08:30:001
1243099112,2cyclictest26061-21tr07:35:231
1243099110,11cyclictest1398-21wc11:15:171
1243099110,10cyclictest25208-21cat09:50:161
1243099110,10cyclictest0-21swapper/109:30:141
1242699119,2cyclictest13731-21grep11:40:180
1242699119,1cyclictest26227-21tr09:50:230
1242699119,1cyclictest2208-21grep10:10:160
1242699119,1cyclictest18589-21grep09:35:170
1242699118,2cyclictest7099-21aten2_r7power_p12:35:140
1242699118,2cyclictest6879-21chrt08:05:120
1242699118,2cyclictest5237-21awk11:20:270
1242699118,2cyclictest3774-21tr09:05:160
1242699118,2cyclictest31002-21grep08:55:120
1242699118,2cyclictest31002-21grep08:55:120
1242699118,2cyclictest29554-21cut11:05:180
1242699118,2cyclictest27363-21cut07:40:140
1242699118,2cyclictest2505-21tr12:25:150
1242699118,2cyclictest24087-21cut10:55:150
1242699118,2cyclictest23692-21wc08:40:000
1242699118,2cyclictest22319-21cut10:50:170
1242699118,2cyclictest20219-21cut11:55:130
1242699118,2cyclictest20150-21sed08:30:190
1242699118,2cyclictest17323-21chrt11:47:090
1242699118,2cyclictest1545-21ls09:00:140
1242699118,2cyclictest13349-21tr09:25:150
1242699118,2cyclictest12629-21cut08:15:180
1242699118,2cyclictest10024-21chrt09:15:280
1242699115,2cyclictest452-21systemd-logind11:00:230
1242699110,9cyclictest0-21swapper/011:30:130
1242699110,4cyclictest0-21swapper/010:45:140
1243799109,1cyclictest9809-21expr10:25:173
1243799109,1cyclictest31481-21expr11:10:143
1243799109,1cyclictest27399-21cat11:00:203
1243799109,1cyclictest22564-21expr07:30:163
1243799109,1cyclictest22320-21perl10:50:173
1243799108,1cyclictest6014-21tr09:10:143
1243799108,1cyclictest31458-21cat12:15:263
1243799108,1cyclictest19242-21grep10:45:153
1243799108,1cyclictest17490-21cat07:20:103
1243799108,1cyclictest13023-21chrt09:25:083
1243799107,3cyclictest39-21ksoftirqd/312:35:193
1243799107,2cyclictest31214-21expr08:55:133
1243799107,2cyclictest31214-21expr08:55:133
1243599109,1cyclictest15785-21cut09:30:162
1243599109,1cyclictest10319-21cat08:10:202
1243599108,2cyclictest24334-21ls10:55:162
1243599108,2cyclictest10118-21taskset11:32:122
1243599108,1cyclictest9713-21tr10:25:172
1243599108,1cyclictest4471-21cut09:05:202
1243599108,1cyclictest3640-21cpu11:20:152
1243599108,1cyclictest32114-21chrt10:05:102
1243599108,1cyclictest29997-21expr10:00:152
1243599108,1cyclictest27599-21grep09:55:152
1243599108,1cyclictest2306-21cat09:00:212
1243599108,1cyclictest20384-21aten2_r7power_v09:40:152
1243599108,1cyclictest17866-21aten2_r7power_e11:50:152
1243599108,1cyclictest16620-21seq10:40:022
1243599108,1cyclictest12627-21grep08:15:182
1243599107,2cyclictest6574-21cron10:20:002
1243599107,2cyclictest27243-21cat09:55:002
1243599107,2cyclictest2187-21basename10:10:162
1243599107,2cyclictest12095-21cat09:20:272
1243599107,2cyclictest11841-21aten2_r7power_c10:30:152
1243599101,5cyclictest24362-21munin-run07:35:012
1243599100,9cyclictest22609-21gpgconf12:00:002
1243599100,9cyclictest0-21swapper/208:20:062
1243099108,2cyclictest7524-21grep09:10:261
1243099108,1cyclictest8059-21grep08:05:241
1243099108,1cyclictest3246-21sed12:25:191
1243099108,1cyclictest24524-21cut08:40:181
1243099108,1cyclictest22329-21grep10:50:181
1243099108,1cyclictest21515-21cut08:35:161
1243099108,1cyclictest193542sleep107:21:531
1243099107,2cyclictest994-21cat09:00:011
1243099107,2cyclictest9567-21tr08:10:141
1243099107,2cyclictest5186-21sed12:30:151
1243099107,2cyclictest4767-21head09:05:221
1243099107,2cyclictest3861-21chrt10:10:381
1243099107,2cyclictest28787-21tr12:10:231
1243099107,2cyclictest2503-21expr07:55:141
1243099107,2cyclictest24314-21head09:45:261
1243099107,2cyclictest23797-21sort12:00:201
1243099107,2cyclictest10253-21chrt11:33:561
1243099106,3cyclictest12575-21chrt09:22:221
1243099100,10cyclictest7005-21grep11:25:201
1243099100,10cyclictest29899-21grep11:05:211
1242699108,1cyclictest970-21sed12:20:210
1242699108,1cyclictest5277-21cut12:30:170
1242699108,1cyclictest256002sleep008:40:340
1242699108,1cyclictest17312-21cut10:40:180
1242699108,1cyclictest16590-21grep09:30:210
1242699108,1cyclictest16324-21tr07:15:220
1242699108,1cyclictest14412sleep007:52:180
1242699108,1cyclictest1356-21sed10:05:280
1242699108,1cyclictest10448-21wc10:25:210
1242699107,2cyclictest29253-21chrt09:57:110
1242699107,2cyclictest25988-21sed12:05:200
1242699107,2cyclictest24273-21chrt07:34:070
1242699107,2cyclictest2408-21grep11:15:240
1242699107,2cyclictest19492-21chrt07:23:410
1242699103,4cyclictest14413-21sshd07:10:330
124359998,1cyclictest5957-21basename09:10:142
124359998,1cyclictest5116-21chrt08:00:172
124359998,1cyclictest3660-21cat07:55:252
124359998,1cyclictest30040-21cat07:45:162
124359998,1cyclictest20797-21taskset10:45:382
124359997,1cyclictest58302sleep211:25:072
124359996,2cyclictest14247-21grep11:40:222
124359996,2cyclictest11379-21head11:35:162
124359993,5cyclictest2253-21ntp_states11:15:242
124359992,4cyclictest23553-21cron08:40:002
124359990,9cyclictest2538-21expr12:25:152
124269998,1cyclictest29256-21grep12:10:280
124269997,1cyclictest6017-21wc11:25:140
124269997,1cyclictest27420-21cut08:45:230
124269997,1cyclictest18593-21tail11:50:180
124269997,1cyclictest125342sleep009:21:490
124269996,2cyclictest29809-21cat07:45:150
124269991,4cyclictest225-21systemd-journal08:00:330
124269986,1cyclictest11339-21grep11:35:160
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional