You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-07 - 18:25

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Mon Sep 07, 2026 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1319921220,0sleep10-21swapper/111:04:491
2199529571,19sleep30-21swapper/307:05:293
2213929076,9sleep20-21swapper/207:05:312
2244728867,6sleep10-21swapper/107:09:311
2097228369,9sleep00-21swapper/007:05:170
22577993127,3cyclictest12693-21kworker/3:011:36:043
2257599290,29cyclictest0-21swapper/211:05:592
2257599290,28cyclictest0-21swapper/211:25:502
2257599290,28cyclictest0-21swapper/210:28:062
2257599280,28cyclictest0-21swapper/210:55:392
2257599250,25cyclictest0-21swapper/211:14:132
2256899250,25cyclictest0-21swapper/012:08:110
2257799242,1cyclictest3511-21kworker/u8:210:45:153
2257599240,24cyclictest0-21swapper/212:36:402
22568992423,1cyclictest0-21swapper/011:20:450
2256899240,24cyclictest0-21swapper/011:26:270
22577992320,3cyclictest12693-21kworker/3:011:27:033
22577992320,2cyclictest12693-21kworker/3:012:12:343
2257799230,22cyclictest12693-21kworker/3:010:30:333
22575992320,2cyclictest32-21ksoftirqd/210:11:092
22577992218,4cyclictest12693-21kworker/3:011:58:333
2257599220,21cyclictest0-21swapper/211:16:142
2257599220,20cyclictest0-21swapper/210:03:452
2257799210,21cyclictest0-21swapper/312:28:093
22577992019,1cyclictest0-21swapper/310:07:003
22577992018,2cyclictest12693-21kworker/3:011:16:533
22577992017,3cyclictest12693-21kworker/3:011:08:333
2257799201,3cyclictest0-21swapper/312:35:263
22575992016,3cyclictest32-21ksoftirqd/212:30:142
2257599200,20cyclictest0-21swapper/210:36:342
2256899206,7cyclictest4620-21munin-run07:40:010
2256899200,1cyclictest0-21swapper/010:28:270
22577991917,2cyclictest12693-21kworker/3:011:42:333
2257199190,18cyclictest0-21swapper/109:48:391
2256899196,6cyclictest9990-21idleruntime-cro12:40:000
22577991816,2cyclictest12693-21kworker/3:011:54:133
22577991816,2cyclictest12693-21kworker/3:011:34:233
22577991816,2cyclictest12693-21kworker/3:010:56:233
22577991815,2cyclictest39-21ksoftirqd/309:22:033
2257799180,17cyclictest0-21swapper/310:20:233
2257599181,17cyclictest32-21ksoftirqd/209:20:552
2257199180,17cyclictest6586-21ssh10:40:351
3602170,0sleep00-21swapper/010:37:260
2257799178,8cyclictest39-21ksoftirqd/307:30:193
2257799174,13cyclictest20788-21kworker/u8:109:34:183
22577991717,0cyclictest0-21swapper/309:59:033
22577991715,2cyclictest12693-21kworker/3:011:13:143
22577991715,2cyclictest0-21swapper/310:39:483
2257799170,11cyclictest0-21swapper/308:30:123
2257599170,17cyclictest0-21swapper/211:21:412
2257199176,10cyclictest15372-21diskmemload11:55:131
2256899173,8cyclictest12041-21perf07:55:010
2256899170,15cyclictest0-21swapper/009:37:370
2257799162,13cyclictest0-21swapper/310:17:523
22577991614,2cyclictest12693-21kworker/3:011:45:333
2257599160,15cyclictest0-21swapper/207:10:202
2257599160,14cyclictest0-21swapper/212:06:572
22571991615,1cyclictest15372-21diskmemload10:16:051
2257199160,16cyclictest0-21swapper/109:18:151
22568991615,1cyclictest0-21swapper/011:49:500
2257799158,7cyclictest1436-21sendmail-mta09:12:283
22577991513,2cyclictest12693-21kworker/3:010:52:133
22577991512,3cyclictest12693-21kworker/3:012:16:033
2257599152,1cyclictest15372-21diskmemload11:50:362
22575991514,1cyclictest0-21swapper/210:18:162
22575991512,2cyclictest31503-21wget11:35:132
2257599150,1cyclictest0-21swapper/209:33:112
2257599150,13cyclictest0-21swapper/212:11:592
2257199150,3cyclictest0-21swapper/111:51:411
2257199150,15cyclictest0-21swapper/111:41:571
22568991514,1cyclictest0-21swapper/012:18:510
22577991412,2cyclictest12693-21kworker/3:011:00:443
22577991410,3cyclictest5392-21chrt08:45:253
2257599147,6cyclictest21496-21munin-node09:39:442
2257599147,6cyclictest20200-21cut08:10:172
2257599147,6cyclictest20200-21cut08:10:162
22575991410,3cyclictest9828-21wc12:00:222
22575991410,3cyclictest16784-21cron08:05:002
2257599140,1cyclictest0-21swapper/209:14:172
2257599140,14cyclictest0-21swapper/210:31:502
2257199146,4cyclictest0-21swapper/111:05:121
2257199141,12cyclictest2767-21grep07:35:141
22571991410,3cyclictest3931-21cat12:35:251
22571991410,3cyclictest27808-21grep08:25:211
22571991410,3cyclictest10732-21tr12:20:201
2257199140,14cyclictest0-21swapper/109:41:021
2257199140,14cyclictest0-21swapper/109:31:311
2256899143,3cyclictest1142-21runrttasks11:07:350
2256899141,7cyclictest14398-21munin-run08:00:000
22568991411,2cyclictest23733-21basename09:14:590
22568991411,2cyclictest11304-21grep10:05:170
22568991410,3cyclictest6313-21grep11:00:160
2257799139,3cyclictest31611-21tr08:35:143
2257799139,2cyclictest19850-21wc09:50:273
22577991313,0cyclictest0-21swapper/310:40:563
22577991311,1cyclictest0-21swapper/311:21:063
22577991310,2cyclictest9618-21wc07:50:013
22577991310,2cyclictest20148-21tail08:10:163
22577991310,2cyclictest20148-21tail08:10:163
22577991310,2cyclictest1568-21tr08:40:153
22577991310,2cyclictest11679-21grep09:45:253
22577991310,2cyclictest11466-21sed12:20:263
2257799130,1cyclictest0-21swapper/309:41:433
2257599139,3cyclictest21815-21basename08:15:142
22575991312,1cyclictest55650irq/127-enp2s0-11:48:112
22575991311,2cyclictest32-21ksoftirqd/210:44:452
22575991310,2cyclictest8791-21grep09:25:152
22575991310,2cyclictest30670-21head08:30:252
22575991310,2cyclictest19933-21grep09:50:272
22575991310,2cyclictest19837-21head12:25:272
22575991310,2cyclictest17541-21turbostat.cron09:50:002
22575991310,2cyclictest11073-21apache207:20:242
2257599130,12cyclictest0-21swapper/207:30:132
2257199139,3cyclictest579-21cron09:40:001
2257199138,3cyclictest544-21cat09:20:151
2257199135,2cyclictest32289-21munin-plugin-st07:30:001
22571991312,1cyclictest2604-21ssh11:36:391
22571991310,2cyclictest3926-21sort10:00:271
22571991310,2cyclictest25-21ksoftirqd/110:47:361
22571991310,2cyclictest23449-21grep08:15:271
22571991310,2cyclictest16051-21cut11:25:201
2257199130,12cyclictest27374-21grep12:30:181
2257199130,11cyclictest0-21swapper/108:45:121
2256899139,3cyclictest4645-21ssh10:20:210
2256899139,3cyclictest4416-21grep08:45:160
2256899131,6cyclictest26338-21munin-run08:25:010
22568991310,2cyclictest7520-21grep11:40:150
22568991310,2cyclictest24202-21basename11:50:140
22568991310,2cyclictest1793-21grep09:40:170
2256899130,1cyclictest9759-21pmu-power12:00:210
2256899130,11cyclictest18271-21grep09:30:220
2256899130,11cyclictest0-21swapper/012:13:150
2257799129,2cyclictest3267-21grep07:35:163
2257799129,2cyclictest25448-21grep07:15:153
2257799129,2cyclictest22599-21head08:15:183
2257799129,2cyclictest17605-21head12:05:173
2257599129,2cyclictest7103-21tr11:00:232
2257599129,2cyclictest5836-21grep07:40:162
2257599129,2cyclictest3207-21tail07:35:162
2257599129,2cyclictest27308-21ls07:19:592
2257599129,2cyclictest26156-21tail09:15:272
2257599129,2cyclictest1940-21tail09:40:162
2257599129,2cyclictest18721-21sed08:05:252
2257599129,2cyclictest1661-21df12:15:152
2257599129,2cyclictest14896-21tail09:05:252
2257599129,2cyclictest11370-21grep09:00:152
2257599129,2cyclictest10088-21tr07:50:152
2257599124,4cyclictest225-21systemd-journal07:49:562
2257599120,2cyclictest0-21swapper/209:56:562
2257599120,12cyclictest0-21swapper/211:57:282
2257599120,12cyclictest0-21swapper/211:44:242
2257199129,2cyclictest8606-21awk07:45:201
2257199129,2cyclictest2358-21chrt10:58:031
2257199129,2cyclictest20553-21tr08:10:211
2257199129,2cyclictest20553-21tr08:10:201
2257199129,2cyclictest1830-21grep12:15:171
2257199129,2cyclictest17771-21head08:05:161
2257199129,2cyclictest11599-21sed07:50:261
2257199128,3cyclictest9860-21grep09:25:211
2257199128,3cyclictest7128-21grep11:20:151
2257199128,3cyclictest11615-21grep09:00:161
2257199128,1cyclictest14386-21ssh10:06:491
2257199126,2cyclictest225-21systemd-journal10:55:011
2257199122,6cyclictest225-21systemd-journal08:00:001
2257199121,1cyclictest8231-21sh10:22:451
2257199120,9cyclictest0-21swapper/109:11:111
2257199120,1cyclictest0-21swapper/110:35:131
2257199120,11cyclictest0-21swapper/111:16:521
2257199120,11cyclictest0-21swapper/108:50:161
2256899129,2cyclictest823-21sshd11:55:170
2256899129,2cyclictest5881-21grep07:40:160
2256899129,2cyclictest4859-21grep11:00:000
2256899129,2cyclictest32185-21wc09:20:010
2256899129,2cyclictest30208-21cut07:25:130
2256899129,2cyclictest27819-21head08:25:210
2256899129,2cyclictest21388-21wc10:30:210
2256899129,2cyclictest2091-21sed09:20:270
2256899129,2cyclictest14711-21grep08:00:120
2256899129,2cyclictest13001-21perf09:04:590
2256899129,2cyclictest11387-21grep12:20:260
2256899129,1cyclictest0-21swapper/009:50:100
303552110,1sleep00-21swapper/008:30:250
2257799119,1cyclictest288462sleep307:20:233
2257799118,2cyclictest31439-21ssh09:19:313
2257799118,2cyclictest27887-21ssh12:30:243
2257799118,2cyclictest24960-21ssh09:35:033
2257799118,2cyclictest24580-21expr08:20:143
2257799118,2cyclictest23591-21head07:10:173
2257799118,2cyclictest21960-21ssh10:11:123
2257799118,2cyclictest14543-21sed08:00:013
2257799118,2cyclictest14053-21chrt09:05:163
2257799118,2cyclictest12809-21chrt09:27:133
2257799118,2cyclictest10831-21ssh12:00:493
2257799117,3cyclictest13626-21sed10:25:283
2257799110,10cyclictest0-21swapper/308:25:143
2257599118,2cyclictest5335-21grep08:45:252
2257599118,2cyclictest30414-21tr07:25:142
2257599118,2cyclictest2572-21wc08:40:242
2257599118,2cyclictest23256-21cut11:30:132
2257599118,2cyclictest20955-21ssh10:50:072
2257599118,2cyclictest14222-21sh10:06:412
2257599118,2cyclictest13256-21expr10:45:142
2257599118,2cyclictest12415-21cut07:55:142
2257599113,5cyclictest8177-21apt08:55:002
2257599111,4cyclictest6439-21runrttasks10:21:212
2257199118,2cyclictest500-21expr07:30:141
2257199118,2cyclictest28896-21ssh11:14:031
2257199118,2cyclictest25930-21head07:15:171
2257199118,2cyclictest25404-21chrt12:29:481
2257199118,2cyclictest24998-21ssh12:10:071
2257199118,2cyclictest24802-21sed11:30:241
2257199118,2cyclictest23995-21rm10:32:011
2257199118,2cyclictest20510-21ssh11:47:351
2257199118,2cyclictest19311-21grep10:10:151
2257199118,2cyclictest17659-21sed12:05:181
2257199118,2cyclictest13060-21grep10:25:231
2257199118,2cyclictest10572-21ssh12:00:371
2257199117,2cyclictest18867-21head09:50:191
2257199117,1cyclictest102732sleep108:56:451
2257199113,7cyclictest27892-21sensors09:55:211
2256899119,1cyclictest10475-21grep09:45:160
2256899118,2cyclictest9374-21ssh09:25:170
2256899118,2cyclictest9347-21ssh10:42:450
2256899118,2cyclictest8025-21df07:45:150
2256899118,2cyclictest7668-21grep08:50:280
2256899118,2cyclictest3824-21ssh11:18:230
2256899118,2cyclictest30225-21ssh10:16:100
2256899118,2cyclictest29429-21taskset07:21:450
2256899118,2cyclictest28946-21ssh09:55:550
2256899118,2cyclictest28049-21grep12:30:240
2256899118,2cyclictest25526-21ssh11:11:240
2256899118,2cyclictest24174-21ssh10:51:170
2256899118,2cyclictest23353-21cat08:15:250
2256899118,2cyclictest21694-21sed08:15:010
2256899118,2cyclictest21694-21sed08:15:000
2256899118,2cyclictest2118-21ssh11:36:160
2256899118,2cyclictest20135-21ssh12:25:400
2256899118,2cyclictest19785-21grep10:10:180
2256899118,2cyclictest17070-21hostname08:05:130
2256899118,2cyclictest16505-21ssh10:46:460
2256899118,2cyclictest1322-21cut08:40:150
2256899114,6cyclictest553-21sshd07:30:130
2257799108,1cyclictest2488-21grep10:00:163
2257799108,1cyclictest184452sleep308:05:233
2257799108,1cyclictest16437-21taskset08:00:363
2257799108,1cyclictest117752sleep307:52:003
2257799107,2cyclictest31331-21ntp_states07:25:213
2257799106,3cyclictest6367-21grep07:40:213
2257799103,3cyclictest225-21systemd-journal08:50:003
2257799100,9cyclictest8659-21expr08:55:133
2257599108,1cyclictest32285-21cut08:35:182
2257599108,1cyclictest26129-21taskset08:22:382
2257599107,2cyclictest9804-21cstates12:20:152
2257599107,2cyclictest28272-21head08:25:262
2257199108,1cyclictest24811-21chrt07:13:481
2257199108,1cyclictest1716-21ls08:40:161
2257199108,1cyclictest16499-21taskset08:01:241
2257199107,2cyclictest28889-21cut07:20:241
2257199107,2cyclictest15048-21taskset09:06:011
2257199100,9cyclictest32349-21load08:35:191
2257199100,1cyclictest0-21swapper/108:30:171
2257199100,0cyclictest0-21swapper/107:40:121
2256899108,1cyclictest9346-21irqstats08:55:170
2256899108,1cyclictest31786-21cat08:35:140
2256899108,1cyclictest24654-21chrt07:11:470
2256899108,1cyclictest22954-21ssh11:30:040
2256899108,1cyclictest2195-21ssh10:00:110
2256899108,1cyclictest13276-21grep09:05:140
225779997,1cyclictest11911-21expr09:00:193
225759996,2cyclictest10042-21sed08:55:242
225719997,1cyclictest24458-21expr08:20:141
225689997,1cyclictest26491-21cut07:15:220
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional