You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-29 - 19:40

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot7s.osadl.org (updated Wed Apr 29, 2026 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
569621170,0sleep10-21swapper/112:35:011
2376229561,30sleep20-21swapper/207:08:452
2357229250,38sleep00-21swapper/007:06:180
2355328768,15sleep10-21swapper/107:06:031
2373628469,9sleep30-21swapper/307:08:263
23945993027,2cyclictest25-21ksoftirqd/110:20:001
23945993027,2cyclictest25-21ksoftirqd/110:20:001
2394399287,20cyclictest1142-21runrttasks10:56:280
2394399275,2cyclictest7470-21apache_processe11:00:110
23950992625,1cyclictest0-21swapper/211:07:572
23953992519,1cyclictest0-21swapper/309:30:433
23953992519,1cyclictest0-21swapper/309:27:493
2394599242,1cyclictest16749-21diskmemload10:49:521
23953992318,1cyclictest0-21swapper/309:22:563
2395399230,2cyclictest0-21swapper/309:18:553
2394599230,0cyclictest0-21swapper/111:45:341
2394399230,4cyclictest0-21swapper/011:38:500
2394599220,1cyclictest0-21swapper/109:24:221
2394399226,10cyclictest13889-21munin-run12:40:010
2394399221,5cyclictest22498-21head12:25:190
2394599210,1cyclictest0-21swapper/111:35:521
2394599210,19cyclictest0-21swapper/108:00:191
2394399213,4cyclictest0-21swapper/012:07:590
2395099200,1cyclictest0-21swapper/212:16:292
2394599200,2cyclictest0-21swapper/109:31:171
2394399200,1cyclictest0-21swapper/011:22:290
2395399192,15cyclictest16749-21diskmemload11:36:343
2395099194,2cyclictest141rcu_preempt10:07:372
23950991917,1cyclictest0-21swapper/209:58:592
23945991919,0cyclictest0-21swapper/110:27:281
2394399194,4cyclictest15859-21awk09:05:210
23950991816,1cyclictest0-21swapper/212:10:162
2395099180,18cyclictest0-21swapper/211:18:402
2395099180,0cyclictest0-21swapper/210:24:542
23945991813,4cyclictest0-21swapper/112:20:131
2394599180,17cyclictest0-21swapper/109:17:521
2394399188,10cyclictest13-21ksoftirqd/010:33:490
2394399180,3cyclictest0-21swapper/008:15:130
2394399180,2cyclictest0-21swapper/010:23:000
2394399180,16cyclictest0-21swapper/007:25:180
2394599175,7cyclictest1079-21apt-get07:30:001
23945991715,1cyclictest0-21swapper/112:11:081
23945991710,6cyclictest26607-21head11:30:161
2394599170,3cyclictest0-21swapper/111:14:371
2394399176,2cyclictest26276-21cron09:35:000
315902160,0sleep30-21swapper/307:25:163
269992160,0sleep30-21swapper/309:54:423
2395399160,1cyclictest0-21swapper/309:39:343
23950991615,1cyclictest0-21swapper/211:46:522
23950991614,1cyclictest23010-21pmu-power12:25:242
2395099160,16cyclictest0-21swapper/210:27:292
2395099160,13cyclictest0-21swapper/210:04:562
2394599160,15cyclictest0-21swapper/110:23:231
2394399169,6cyclictest9106-21sed08:50:260
2394399168,7cyclictest4142-21sh12:33:570
2394399161,7cyclictest11662-21ssh10:04:570
2394399160,2cyclictest0-21swapper/010:36:100
23953991512,2cyclictest5959-21cat12:35:113
2395399150,1cyclictest0-21swapper/310:07:593
2395399150,15cyclictest0-21swapper/310:41:183
2395099150,15cyclictest0-21swapper/210:40:202
23945991512,2cyclictest25-21ksoftirqd/108:40:011
2394599150,14cyclictest0-21swapper/109:45:391
2394399159,5cyclictest16470-21grep11:05:150
2394399159,4cyclictest10484-21cut08:55:170
2394399158,6cyclictest7682-21sh09:42:420
2394399158,6cyclictest3920-21cut11:55:160
2394399158,6cyclictest29949-21tr07:20:180
2394399158,6cyclictest29076-21grep11:50:260
2394399158,6cyclictest20559-21chrt10:48:370
2394399158,6cyclictest18433-21cat09:10:200
2394399158,6cyclictest10238-21ssh09:25:150
2394399157,7cyclictest10821-21munin-run07:50:000
2394399152,6cyclictest13199-21awk07:55:000
2394399150,8cyclictest8455-21awk07:45:000
2394399150,1cyclictest0-21swapper/012:16:360
23953991410,3cyclictest17465-21sed08:00:253
23953991410,2cyclictest2294-21grep08:40:013
2395399140,1cyclictest0-21swapper/312:23:263
2395399140,13cyclictest0-21swapper/310:18:423
2395399140,13cyclictest0-21swapper/310:18:413
23950991412,2cyclictest32-21ksoftirqd/210:39:482
23950991411,2cyclictest27474-21grep11:50:152
2395099140,11cyclictest0-21swapper/211:27:142
2394599148,5cyclictest25321-21cron11:30:001
2394599148,5cyclictest19432-21ls08:05:221
23945991412,2cyclictest0-21swapper/111:09:171
2394599140,3cyclictest0-21swapper/110:55:121
2394599140,14cyclictest0-21swapper/111:04:281
2394399148,5cyclictest8691-21ssh10:40:520
2394399148,5cyclictest2957-21ssh11:15:500
2394399148,5cyclictest19511-21taskset11:25:310
2394399148,5cyclictest17538-21grep08:00:260
2394399148,5cyclictest12925-21cut10:05:170
2394399147,2cyclictest27428-21wc07:15:180
2394399147,2cyclictest15089-21ssh10:25:250
2394399146,2cyclictest23200-21grep10:50:170
2394399143,4cyclictest18266-21idleruntime-cro11:45:000
2394399142,2cyclictest225-21systemd-journal09:04:590
2394399141,2cyclictest225-21systemd-journal08:14:590
2394399140,5cyclictest225-21systemd-journal11:49:590
2394399140,4cyclictest225-21systemd-journal12:05:010
2395399139,3cyclictest3658-21cron07:34:593
2395399139,3cyclictest26737-21ssh11:10:373
2395399133,7cyclictest18905-21cut11:45:143
2395399131,4cyclictest225-21systemd-journal12:20:003
23953991310,2cyclictest7452-21sh11:00:103
23953991310,2cyclictest31191-21grep07:25:003
23953991310,2cyclictest24524-21tr07:10:143
23953991310,2cyclictest18979-21sed08:05:183
23953991310,2cyclictest17738-21grep11:05:253
23953991310,2cyclictest15591-21tr07:59:593
23950991312,1cyclictest0-21swapper/209:19:422
23950991310,2cyclictest7806-21cat07:40:272
23950991310,2cyclictest3582-21ssh09:40:172
23950991310,2cyclictest16230-21grep09:05:252
23950991310,2cyclictest13760-21cat09:00:252
23950991310,2cyclictest11324-21grep07:50:142
2395099130,2cyclictest0-21swapper/209:47:202
2395099130,12cyclictest3552-21grep09:20:252
2395099130,0cyclictest0-21swapper/210:47:342
2394599139,3cyclictest3334-21grep08:40:161
2394599139,3cyclictest30657-21tail07:20:241
2394599139,3cyclictest12498-21tr09:00:151
23945991312,1cyclictest0-21swapper/111:55:011
23945991310,2cyclictest19756-21dpkg12:05:011
2394399139,2cyclictest2432-21cat08:40:010
2394399138,4cyclictest12429-21ntpq09:45:210
2394399138,1cyclictest31194-21ssh09:37:160
2394399137,5cyclictest19213-21chrt09:50:020
2394399137,2cyclictest27711-21ssh09:55:050
2394399136,2cyclictest29971-21grep10:15:170
2394399136,2cyclictest29971-21grep10:15:170
2394399136,2cyclictest14039-21cut12:20:180
2394399136,1cyclictest2807-21ps07:30:190
2394399134,2cyclictest4176-21tr07:35:110
2394399133,9cyclictest25678-21apt-get09:15:130
2394399133,3cyclictest1550-21ssh09:20:010
2394399131,2cyclictest225-21systemd-journal11:14:500
2394399131,2cyclictest225-21systemd-journal08:35:000
23943991310,2cyclictest22834-21ssh10:10:270
23943991310,2cyclictest15092-21grep07:55:270
2395399129,2cyclictest9585-21cron08:55:013
2395399129,2cyclictest528-21ssh10:55:223
2395399129,2cyclictest5081-21tail10:00:203
2395399129,2cyclictest31735-21grep10:35:213
2395399129,2cyclictest16251-21grep09:05:253
2395399129,2cyclictest13026-21sed09:00:173
2395399128,3cyclictest26133-21grep11:30:133
2395399120,1cyclictest0-21swapper/310:10:313
2395399120,11cyclictest30829-21awk10:55:003
2395099129,2cyclictest6290-21grep12:35:152
2395099129,2cyclictest5345-21cat07:35:232
2395099129,2cyclictest31505-21seq11:14:212
2395099129,2cyclictest25176-21basename08:20:002
2395099129,2cyclictest24883-21grep07:10:162
2395099129,2cyclictest21936-21tr10:30:152
2395099129,2cyclictest21021-21tr10:10:142
2395099129,2cyclictest17969-21sort09:10:162
2395099129,2cyclictest10172-21grep07:45:242
2395099120,4cyclictest0-21swapper/211:35:142
2394599129,2cyclictest8941-21tr07:45:131
2394599129,2cyclictest8129-21cut08:50:161
2394599129,2cyclictest7350-21grep10:40:191
2394599129,2cyclictest27959-21wc09:35:191
2394599129,2cyclictest27399-21sh09:55:001
2394599129,2cyclictest23568-21head10:30:261
2394599129,2cyclictest23021-21basename08:15:131
2394599129,2cyclictest10577-21grep08:55:171
2394599128,3cyclictest24373-21grep10:50:241
2394599124,7cyclictest38150irq/131-snd_hda09:05:121
2394599124,7cyclictest24276-21apache209:12:321
23945991211,1cyclictest16749-21diskmemload12:07:581
2394599120,1cyclictest16749-21diskmemload09:44:361
2394599120,1cyclictest0-21swapper/112:19:391
2394599120,12cyclictest0-21swapper/107:36:531
2394599120,11cyclictest11167-21grep11:40:171
2394399129,2cyclictest25530-21grep08:20:130
2394399129,2cyclictest24985-21tail07:10:150
2394399127,4cyclictest296842sleep008:27:160
2394399127,4cyclictest296842sleep008:27:160
2394399126,1cyclictest6901-21chrt08:46:030
2394399125,2cyclictest29531-21cut12:10:160
2394399121,2cyclictest0-21swapper/011:32:040
2394399120,2cyclictest0-21swapper/008:05:130
2395399119,1cyclictest9724-21sort11:20:153
2395399119,1cyclictest3494-21head08:40:163
2395399118,2cyclictest9077-21expr07:45:143
2395399118,2cyclictest5646-21sh09:41:063
2395399118,2cyclictest5100-21grep11:55:263
2395399118,2cyclictest439-21ssh11:15:143
2395399118,2cyclictest32078-21taskset12:11:263
2395399118,2cyclictest29766-21ssh11:50:523
2395399118,2cyclictest29667-21chrt08:27:033
2395399118,2cyclictest29667-21chrt08:27:023
2395399118,2cyclictest28591-21taskset07:18:243
2395399118,2cyclictest27632-21ssh09:55:003
2395399118,2cyclictest23673-21head08:15:163
2395399118,2cyclictest23020-21grep12:25:233
2395399118,2cyclictest20010-21ssh09:11:013
2395399118,2cyclictest14612-21aten2_r7power_v10:45:133
2395399118,2cyclictest13804-21expr10:25:143
2395399118,2cyclictest12319-21sed11:40:253
2395399118,2cyclictest11826-21grep09:45:173
2395399118,2cyclictest11623-21ssh12:00:073
2395399114,3cyclictest20359-21apt-get12:05:133
2395399110,10cyclictest7381-21sh10:20:373
2395399110,10cyclictest5898-21head08:45:173
2395399110,10cyclictest11988-21basename09:00:013
2395099118,2cyclictest9503-21ssh11:00:262
2395099118,2cyclictest755-21swap10:55:252
2395099118,2cyclictest5374-21expr08:45:142
2395099118,2cyclictest4347-21grep08:40:262
2395099118,2cyclictest29390-21sh10:53:542
2395099118,2cyclictest27287-21chrt08:21:572
2395099118,2cyclictest26790-21ssh09:35:122
2395099118,2cyclictest2391-21ssh12:32:352
2395099118,2cyclictest21828-21ssh09:31:342
2395099118,2cyclictest20331-21cat12:05:112
2395099118,2cyclictest1821-21sed08:35:252
2395099118,2cyclictest1786-21ssh10:17:482
2395099118,2cyclictest1786-21ssh10:17:472
2395099118,2cyclictest16690-21ssh12:02:432
2395099118,2cyclictest15483-21ssh12:20:412
2395099118,2cyclictest14021-21ssh09:26:582
2395099118,2cyclictest13711-21ssh11:41:292
2395099111,9cyclictest13438-21ssh11:22:192
2395099110,3cyclictest0-21swapper/209:51:442
2395099110,2cyclictest0-21swapper/211:56:042
2395099110,10cyclictest28447-21ssh11:30:542
2394599118,2cyclictest32735-21ssh11:15:071
2394599118,2cyclictest30894-21head08:30:161
2394599118,2cyclictest30802-21ssh09:56:091
2394599118,2cyclictest30287-21cut10:35:131
2394599118,2cyclictest30133-21ssh11:51:081
2394599118,2cyclictest25-21ksoftirqd/110:00:261
2394599118,2cyclictest21356-21sh12:25:031
2394599118,2cyclictest20592-21apt-get08:10:121
2394599118,2cyclictest19054-21ssh10:09:111
2394599118,2cyclictest1563-21chrt07:30:111
2394599118,2cyclictest13881-21ssh09:26:511
2394599118,2cyclictest12752-21grep07:50:251
2394599118,2cyclictest10684-21wc11:20:231
2394599117,3cyclictest7553-21ssh12:35:231
2394599111,2cyclictest225-21systemd-journal07:15:001
2395399108,1cyclictest31727-21sed08:30:253
2395399108,1cyclictest22278-21ssh10:30:173
2395399107,2cyclictest4347-21cat07:35:123
2395399107,2cyclictest27185-21chrt08:20:383
2395399107,2cyclictest22300-21sed08:10:263
2395399107,2cyclictest17205-21ssh11:25:053
2395399100,9cyclictest29926-21ssh12:30:073
2395099108,1cyclictest29848-21basename07:20:162
2395099108,1cyclictest21372-21grep08:10:172
2395099108,1cyclictest1957-21expr07:30:132
2395099108,1cyclictest14237-21smartctl07:55:182
2395099107,2cyclictest31527-21grep07:25:162
2395099107,2cyclictest18313-21cut08:05:152
2395099107,2cyclictest16381-21df_abs08:00:152
2395099106,3cyclictest10118-21tr08:55:152
2395099105,2cyclictest225-21systemd-journal07:20:002
2395099100,9cyclictest94962sleep208:53:542
2395099100,9cyclictest30132-21chrt08:30:022
2395099100,9cyclictest28628-21basename08:25:182
2395099100,9cyclictest28628-21basename08:25:172
2394599108,1cyclictest8081-21taskset07:40:361
2394599108,1cyclictest20618-21ssh10:10:041
2394599108,1cyclictest154742sleep107:59:081
2394599107,2cyclictest27532-21cron08:24:591
2394399107,2cyclictest3002-21cut08:40:130
239539997,1cyclictest8491-21cat07:45:003
239539997,1cyclictest11423-21expr07:50:143
239459997,1cyclictest6350-21proc_pri08:45:221
239459997,1cyclictest27817-21cut08:25:141
239459997,1cyclictest27817-21cut08:25:141
239459991,1cyclictest0-21swapper/107:15:141
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional