You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-04 - 19:35

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot7s.osadl.org (updated Thu Jun 04, 2026 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3043929986,9sleep00-21swapper/007:05:450
3051329164,9sleep20-21swapper/207:06:442
3063628876,8sleep30-21swapper/307:08:223
3046028867,6sleep10-21swapper/107:06:021
3086099280,28cyclictest0-21swapper/212:32:322
3086099280,28cyclictest0-21swapper/209:41:082
3086099280,28cyclictest0-21swapper/209:41:072
3086099270,25cyclictest0-21swapper/211:28:212
3085799263,22cyclictest23919-21diskmemload11:05:171
82262250,1sleep17828-21kworker/1:207:29:551
3086099250,25cyclictest0-21swapper/212:03:072
3086099250,25cyclictest0-21swapper/209:52:182
3086099240,21cyclictest0-21swapper/210:21:452
30857992421,1cyclictest0-21swapper/110:50:171
3086099230,23cyclictest0-21swapper/209:32:382
3086099230,22cyclictest0-21swapper/211:45:132
3085499230,2cyclictest0-21swapper/009:16:480
3086099220,22cyclictest0-21swapper/211:19:322
3086099220,20cyclictest0-21swapper/210:59:332
3085799222,1cyclictest141rcu_preempt11:47:171
3085799220,20cyclictest0-21swapper/107:55:161
30860992121,0cyclictest0-21swapper/211:53:032
3086099210,20cyclictest0-21swapper/210:40:462
3086099210,1cyclictest0-21swapper/209:13:252
3086099210,19cyclictest0-21swapper/210:04:562
3086099200,20cyclictest0-21swapper/210:10:172
3086099200,19cyclictest0-21swapper/211:10:052
3086099200,19cyclictest0-21swapper/210:30:112
3086099200,18cyclictest0-21swapper/212:24:342
204022200,0sleep10-21swapper/111:03:161
3086399190,17cyclictest0-21swapper/310:18:013
3086099191,2cyclictest0-21swapper/209:46:132
3086099190,16cyclictest0-21swapper/209:28:332
30857991917,1cyclictest0-21swapper/110:30:131
3085799190,3cyclictest28682-21df_abs12:25:131
3085799190,1cyclictest0-21swapper/111:50:111
3085499196,7cyclictest0-21swapper/012:25:160
3085499190,17cyclictest0-21swapper/012:38:460
3086399181,2cyclictest0-21swapper/310:30:133
3086099180,17cyclictest0-21swapper/211:57:462
3085799181,17cyclictest23919-21diskmemload12:04:111
3085799180,17cyclictest0-21swapper/111:25:111
30854991812,4cyclictest31534-21ls10:50:180
3085499180,1cyclictest0-21swapper/010:16:540
30863991715,1cyclictest0-21swapper/311:10:443
30863991712,1cyclictest39-21ksoftirqd/311:50:153
30857991713,2cyclictest2675-21kworker/1:210:21:421
3085799170,1cyclictest23919-21diskmemload11:22:101
30854991717,0cyclictest0-21swapper/011:42:410
3086399161,14cyclictest23984-21hddtemp_smartct11:05:173
30863991611,2cyclictest3582-21cron12:30:003
3086099160,3cyclictest0-21swapper/212:25:452
3086099160,16cyclictest0-21swapper/209:56:172
3086099160,16cyclictest0-21swapper/209:18:112
3086099160,15cyclictest0-21swapper/207:35:112
30857991615,1cyclictest23919-21diskmemload10:40:121
3085799160,1cyclictest0-21swapper/109:40:221
3085799160,1cyclictest0-21swapper/109:40:211
3085799160,14cyclictest0-21swapper/110:19:591
3085799160,14cyclictest0-21swapper/109:10:411
3085499163,7cyclictest20366-21awk07:55:010
3085499160,16cyclictest0-21swapper/011:28:280
30860991515,0cyclictest0-21swapper/210:45:052
3086099151,12cyclictest0-21swapper/209:20:182
3086099150,15cyclictest0-21swapper/211:39:212
3085799159,2cyclictest28681-21cat09:50:251
30857991514,1cyclictest0-21swapper/111:30:251
3085799150,2cyclictest0-21swapper/110:05:181
3085799150,15cyclictest0-21swapper/112:20:501
3085499152,11cyclictest537-21snmpd12:22:400
30854991514,1cyclictest0-21swapper/011:50:120
30854991511,3cyclictest21560-21cron10:45:000
3085499151,13cyclictest11599-21ssh10:00:120
30854991510,4cyclictest7299-21ls07:25:260
3085499150,3cyclictest0-21swapper/012:17:350
3085499150,15cyclictest0-21swapper/012:13:440
3086399146,6cyclictest10863-21fschecks_time11:55:173
3086399145,3cyclictest21145-21/usr/sbin/munin10:25:153
3086099148,5cyclictest13737-21munin-node12:38:452
3086099148,5cyclictest13737-21munin-node09:36:082
3086099148,5cyclictest13737-21munin-node09:36:082
30860991410,2cyclictest18612-21wc11:40:212
3086099140,14cyclictest0-21swapper/211:30:492
3086099140,13cyclictest0-21swapper/207:45:172
3086099140,13cyclictest0-21swapper/207:45:162
3085799142,3cyclictest3469-21expr09:55:151
3085799141,11cyclictest7023-21df_inode10:55:151
3085799140,1cyclictest0-21swapper/109:39:091
3085799140,1cyclictest0-21swapper/109:39:081
3085499143,6cyclictest27025-21ssh09:50:120
30854991413,1cyclictest0-21swapper/012:05:290
30854991413,1cyclictest0-21swapper/009:48:090
30854991410,3cyclictest26512-21sh09:10:300
30854991410,2cyclictest23116-21grep08:00:150
30854991410,2cyclictest20718-21grep07:55:140
93162130,3sleep30-21swapper/310:36:593
3086399138,5cyclictest1370-21apache210:51:053
3086399135,6cyclictest32732-21kworker/u8:009:15:513
3086399130,9cyclictest0-21swapper/309:56:493
3086399130,1cyclictest1405-21kworker/3:011:21:203
3086399130,12cyclictest0-21swapper/310:45:133
3086399130,11cyclictest0-21swapper/309:24:043
3086099139,3cyclictest28992-21sed12:05:272
3086099139,3cyclictest1877-21wc08:20:252
3086099137,2cyclictest225-21systemd-journal11:25:012
3086099134,2cyclictest225-21systemd-journal11:03:112
30860991310,2cyclictest12099-21cron10:20:002
3086099130,11cyclictest0-21swapper/210:36:132
3085799139,3cyclictest29979-21wc10:10:251
3085799138,4cyclictest38150irq/131-snd_hda09:04:591
3085799137,5cyclictest1416-21cat09:15:181
3085799132,10cyclictest537-21snmpd09:21:361
30857991311,1cyclictest14210-21ssh11:18:581
30857991310,2cyclictest4874-21head12:30:151
3085799130,4cyclictest0-21swapper/112:18:001
3085799130,1cyclictest0-21swapper/110:45:041
3085799130,10cyclictest0-21swapper/111:58:501
3085499131,6cyclictest27516-21awk08:10:010
3085499131,5cyclictest3424-21cups-browsed09:25:210
30854991310,2cyclictest8388-21cut11:15:140
30854991310,2cyclictest30107-21grep10:10:260
30854991310,2cyclictest20213-21cat12:00:260
30854991310,2cyclictest1973-21cstates09:35:150
30854991310,2cyclictest1973-21cstates09:35:140
30854991310,2cyclictest14584-21tail07:40:200
30854991310,2cyclictest14323-21sh11:00:000
3085499130,7cyclictest17999-21perf07:50:010
3085499130,7cyclictest17999-21perf07:50:010
3085499130,0cyclictest0-21swapper/010:22:540
3086399129,2cyclictest31119-21ssh09:33:153
3086399129,2cyclictest30392-21cut08:15:133
3086399129,2cyclictest27451-21wc11:45:243
3086399129,2cyclictest26774-21rm12:05:013
3086399129,2cyclictest25790-21grep11:25:183
3086399129,2cyclictest25302-21ssh09:10:163
3086399129,2cyclictest20276-21perf12:39:593
3086399129,2cyclictest18772-21tr12:00:133
3086399129,2cyclictest18488-21tr07:50:133
3086399129,2cyclictest12252-21grep12:15:173
3086399129,2cyclictest11711-21sh10:58:023
3086399129,2cyclictest11292-21sed10:00:013
30863991211,1cyclictest39-21ksoftirqd/310:20:543
3086399121,11cyclictest20724-21expr10:05:173
30863991210,1cyclictest8158-21cut08:35:173
30863991210,1cyclictest0-21swapper/309:29:253
3086399120,12cyclictest0-21swapper/312:33:193
3086399120,11cyclictest31096-21ssh09:52:053
3086099129,2cyclictest3817-21ssh12:10:182
3086099129,2cyclictest20840-21wc09:00:232
3086099129,2cyclictest19010-21sed07:50:162
3086099129,2cyclictest12010-21cron08:45:002
3086099128,3cyclictest22315-21grep07:55:262
30860991211,1cyclictest23919-21diskmemload10:28:192
3086099120,0cyclictest0-21swapper/211:09:182
3085799129,2cyclictest8382-21ssh10:36:131
3085799129,2cyclictest32747-21sed07:10:261
3085799127,5cyclictest3398-21apache212:09:161
3085799124,6cyclictest3398-21apache209:31:031
3085799122,1cyclictest0-21swapper/111:41:561
30857991210,1cyclictest27509-21perf08:10:011
3085799120,8cyclictest0-21swapper/112:10:291
3085799120,12cyclictest23919-21diskmemload10:28:201
3085799120,12cyclictest0-21swapper/111:12:281
3085799120,11cyclictest7565-21hostname08:35:131
3085799120,10cyclictest0-21swapper/108:10:131
3085499129,2cyclictest8519-21dump-pmu-power11:35:010
3085499129,2cyclictest3746-21tr07:20:130
3085499129,2cyclictest30270-21wc08:15:000
3085499129,2cyclictest23514-21wc10:45:220
3085499129,2cyclictest13046-21wc07:40:010
3085499129,2cyclictest1265-21tr07:15:140
3085499128,3cyclictest15887-21grep08:50:210
3085499127,2cyclictest4485-21cat12:30:130
3085499124,7cyclictest12039-21perf08:45:000
3085499123,8cyclictest2969-21ssh09:55:140
3085499120,1cyclictest0-21swapper/011:08:450
3085499120,11cyclictest9673-21cat08:39:590
3085499120,11cyclictest1087-21grep08:20:170
3086399119,2cyclictest4021-21sed12:10:183
3086399119,2cyclictest1982-21chrt09:35:153
3086399119,2cyclictest1982-21chrt09:35:153
3086399119,2cyclictest14684-21ssh11:19:213
3086399119,1cyclictest9810-21head11:35:143
3086399119,1cyclictest9267-21sed07:30:173
3086399119,1cyclictest76632sleep307:26:003
3086399119,1cyclictest426-21wget08:20:133
3086399119,1cyclictest3568-21cut07:20:133
3086399119,1cyclictest26646-21wc08:05:223
3086399119,1cyclictest24861-21ssh12:22:523
3086399119,1cyclictest226202sleep307:58:493
3086399119,1cyclictest22082-21cut09:05:143
3086399119,1cyclictest214722sleep309:04:213
3086399119,1cyclictest17904-21ssh10:42:093
3086399119,1cyclictest1169-21cut07:15:153
3086399119,1cyclictest116522sleep308:40:253
3086399118,2cyclictest714-21cut11:30:153
3086399118,2cyclictest29984-21chrt08:13:533
3086399118,2cyclictest28513-21tr10:10:143
3086399118,2cyclictest24386-21ssh11:44:193
3086399118,2cyclictest16414-21wc11:00:243
3086399118,2cyclictest14915-21tr08:50:143
3086399118,2cyclictest14711-21grep07:40:223
3086399118,2cyclictest12168-21ssh09:40:303
3086399118,2cyclictest12168-21ssh09:40:293
3086399117,2cyclictest19469-21irqstats09:45:183
3086099118,2cyclictest5886-21cut08:30:162
3086099118,2cyclictest5418-21ssh10:54:382
3086099118,2cyclictest4588-21tail07:20:182
3086099118,2cyclictest32188-21head07:10:202
3086099118,2cyclictest20748-21tr10:05:172
3086099118,2cyclictest18813-21chrt08:55:552
3086099118,2cyclictest15605-21ssh12:17:062
3086099118,2cyclictest1270-21grep07:15:142
3086099118,2cyclictest12561-21tr08:45:132
3086099117,3cyclictest26054-21grep08:05:162
3086099112,8cyclictest0-21swapper/207:30:142
3085799119,2cyclictest12223-21ssh11:36:221
3085799119,2cyclictest11960-21expr10:00:131
3085799119,1cyclictest31278-21uname08:15:181
3085799119,1cyclictest3071-21chrt07:19:041
3085799119,1cyclictest19159-21systemctl09:00:001
3085799119,1cyclictest16622-21head07:45:171
3085799119,1cyclictest16622-21head07:45:161
3085799119,1cyclictest1112-21head08:20:161
3085799118,2cyclictest9872-21taskset08:40:061
3085799118,2cyclictest4146-21grep07:20:151
3085799118,2cyclictest19082-21ssh09:45:161
3085799117,4cyclictest10297-21taskset07:30:591
3085799116,4cyclictest25325-21cut08:05:011
3085799110,10cyclictest16757-21ssh09:25:021
3085499118,2cyclictest9987-21sh09:20:200
3085499118,2cyclictest7040-21ssh11:14:550
3085499118,2cyclictest6130-21ls08:30:180
3085499118,2cyclictest5226-21ssh10:35:060
3085499118,2cyclictest31105-21wc10:30:250
3085499118,2cyclictest29100-21ssh09:31:400
3085499118,2cyclictest25932-21basename11:45:140
3085499118,2cyclictest23027-21ssh10:06:180
3085499118,2cyclictest19715-21cut09:00:120
3085499118,2cyclictest16603-21grep11:20:150
3085499118,2cyclictest16249-21wc11:00:200
3085499118,2cyclictest13437-21ssh11:56:270
3086399109,1cyclictest141772sleep308:47:223
3086399108,1cyclictest2674-21chrt08:25:113
3086399108,1cyclictest229702sleep308:00:063
3086399108,1cyclictest158322sleep307:45:083
3086399108,1cyclictest158322sleep307:45:083
3086399107,2cyclictest17278-21tr08:55:133
3086399100,4cyclictest0-21swapper/308:30:113
3086399100,10cyclictest32125-21sed07:10:203
3086099108,1cyclictest30401-21grep08:15:132
3086099107,2cyclictest9235-21head08:35:272
3086099107,2cyclictest4699-21chrt08:27:542
3086099107,2cyclictest28711-21grep08:10:172
3086099107,2cyclictest23149-21wget08:00:162
3086099107,2cyclictest22571-21grep09:05:162
3086099107,2cyclictest14885-21tr08:50:142
3086099107,2cyclictest14172-21sort07:40:182
3086099103,2cyclictest225-21systemd-journal07:27:292
3085799109,1cyclictest3113-21cat08:25:151
3085799109,1cyclictest19987-21taskset07:50:291
3085799109,1cyclictest14756-21processes07:40:221
3085799109,1cyclictest12862-21dnf08:45:141
3085799108,1cyclictest6368-21grep08:30:201
3085799108,1cyclictest12204-21sh12:35:031
3085799107,2cyclictest15530-21if_enp2s008:50:171
3085499108,1cyclictest3505-21grep08:25:170
3085499108,1cyclictest238512sleep009:09:230
3085499108,1cyclictest104212sleep007:32:350
3085499107,2cyclictest9863-21ssh09:40:040
3085499107,2cyclictest9863-21ssh09:40:040
3085499107,2cyclictest13019-21grep08:45:160
3085499103,3cyclictest3391-21apache208:55:110
3085499100,9cyclictest8782-21ssh11:35:030
3085499100,9cyclictest20859-21rm10:25:070
308639997,1cyclictest10888-21apt-get07:35:103
308579998,1cyclictest217622sleep109:05:041
308579997,1cyclictest10880-21apache_accesses07:35:101
308549997,1cyclictest434-21taskset07:11:210
308549996,2cyclictest30626-21expr08:15:140
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional