You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-10 - 22:35

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot7s.osadl.org (updated Sat Jan 10, 2026 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1027829682,10sleep00-21swapper/007:07:530
1014729079,6sleep20-21swapper/207:06:092
1032328961,9sleep30-21swapper/307:08:273
1034128164,13sleep10-21swapper/107:08:421
1054699290,28cyclictest0-21swapper/210:58:252
27102250,0sleep10-21swapper/111:38:551
10540992524,1cyclictest0-21swapper/012:29:030
1055199240,24cyclictest0-21swapper/310:49:133
10540992422,1cyclictest27058-21cat11:15:190
1054099240,4cyclictest0-21swapper/010:12:200
50192220,0sleep10-21swapper/110:44:151
1054099219,11cyclictest0-21swapper/010:20:150
1054099210,1cyclictest0-21swapper/011:49:470
1054099210,1cyclictest0-21swapper/009:27:300
1054699207,7cyclictest225-21systemd-journal09:54:592
1054699200,2cyclictest0-21swapper/211:48:142
1054699191,8cyclictest0-21swapper/207:50:132
1054699190,19cyclictest0-21swapper/212:29:032
10543991917,1cyclictest0-21swapper/112:10:131
1055199180,1cyclictest0-21swapper/309:26:283
1054699183,1cyclictest0-21swapper/211:53:232
1054699180,1cyclictest0-21swapper/209:24:302
1054699180,16cyclictest0-21swapper/212:18:512
10543991818,0cyclictest0-21swapper/111:32:541
10543991813,4cyclictest38150irq/131-snd_hda08:15:121
10543991812,3cyclictest0-21swapper/109:42:081
10543991811,4cyclictest0-21swapper/111:05:261
10551991715,1cyclictest4488-21grep10:25:133
1054399170,1cyclictest0-21swapper/110:33:331
10551991615,1cyclictest0-21swapper/309:34:303
1054699160,2cyclictest0-21swapper/209:38:432
10543991612,4cyclictest25-21ksoftirqd/109:27:471
1054099160,7cyclictest0-21swapper/007:50:130
10551991512,3cyclictest0-21swapper/311:30:273
10546991514,1cyclictest32-21ksoftirqd/211:25:592
1054699151,14cyclictest0-21swapper/211:18:412
1054699150,2cyclictest0-21swapper/209:27:292
1054399155,3cyclictest38150irq/131-snd_hda11:19:571
10543991514,1cyclictest3152-21diskmemload11:54:361
1054399150,1cyclictest225-21systemd-journal10:25:001
1054399150,14cyclictest0-21swapper/110:15:131
1054399150,14cyclictest0-21swapper/110:15:121
1054399150,12cyclictest0-21swapper/111:10:111
10540991513,2cyclictest0-21swapper/012:00:240
1054099151,2cyclictest0-21swapper/009:55:120
1055199149,4cyclictest14953-21cat11:45:233
1055199147,6cyclictest16651-21munin-run09:55:003
10551991411,3cyclictest537-21snmpd11:12:333
10551991411,2cyclictest1221-21wc11:00:203
10551991410,3cyclictest30987-21cat09:00:013
1055199140,13cyclictest23041-21grep07:35:143
10546991412,2cyclictest32-21ksoftirqd/209:57:202
1054699141,10cyclictest0-21swapper/211:05:142
1054699140,1cyclictest0-21swapper/210:35:002
1054699140,0cyclictest0-21swapper/210:22:542
10543991410,2cyclictest3500-21tail12:35:181
1054399140,14cyclictest0-21swapper/112:30:551
1054399140,13cyclictest0-21swapper/112:05:561
1054399140,13cyclictest0-21swapper/112:05:551
1054099149,4cyclictest21841-21apache209:17:560
1054099146,6cyclictest21841-21apache211:05:080
1054099141,12cyclictest23565-21ssh09:40:110
1054099141,11cyclictest0-21swapper/009:51:000
1054099140,12cyclictest0-21swapper/011:54:350
1054099140,10cyclictest0-21swapper/011:00:250
1055199139,3cyclictest29341-21wc10:20:183
1055199139,3cyclictest16523-21wc08:30:013
1055199136,6cyclictest4157-21munin-node11:52:473
10551991312,1cyclictest55650irq/127-enp2s0-11:59:213
10551991311,2cyclictest39-21ksoftirqd/309:43:253
10551991310,2cyclictest5595-21sort11:40:183
10551991310,2cyclictest32316-21grep10:40:273
10551991310,2cyclictest30389-21sed11:35:253
10551991310,2cyclictest21234-21head07:30:173
10551991310,2cyclictest18821-21cron08:35:013
10551991310,2cyclictest17395-21sed07:20:273
1055199130,9cyclictest0-21swapper/307:45:143
1055199130,12cyclictest19390-21chrt10:15:113
1055199130,12cyclictest19390-21chrt10:15:113
1054699139,3cyclictest21471-21ls11:30:262
1054699138,4cyclictest27283-21ntp_states10:00:232
1054699133,5cyclictest15483-21munin-run07:20:002
10546991310,2cyclictest29373-21grep08:55:162
10546991310,2cyclictest26039-21grep07:40:162
1054699130,3cyclictest0-21swapper/211:35:332
1054699130,3cyclictest0-21swapper/210:18:402
1054699130,3cyclictest0-21swapper/210:18:392
1054699130,2cyclictest0-21swapper/210:05:022
1054699130,1cyclictest0-21swapper/212:09:042
1054699130,1cyclictest0-21swapper/212:09:032
1054699130,13cyclictest0-21swapper/211:42:052
1054699130,13cyclictest0-21swapper/210:29:392
1054399139,3cyclictest32723-21grep09:00:261
1054399139,3cyclictest29383-21grep08:55:161
1054399139,2cyclictest28594-21wc08:55:001
1054399139,2cyclictest19196-21cut08:35:131
1054399139,2cyclictest17816-21taskset09:36:081
1054399138,4cyclictest38150irq/131-snd_hda12:00:011
1054399134,9cyclictest0-21swapper/110:08:331
10543991310,2cyclictest5711-21grep10:25:201
10543991310,2cyclictest4341-21sort09:10:161
10543991310,2cyclictest27551-21wc07:45:001
10543991310,2cyclictest23407-21ssh12:28:111
10543991310,2cyclictest14904-21ssh10:50:011
1054399130,1cyclictest0-21swapper/111:41:241
1054099132,10cyclictest1142-21runrttasks12:14:520
1054099130,9cyclictest0-21swapper/012:30:020
1054099130,11cyclictest0-21swapper/007:10:000
1054099130,10cyclictest0-21swapper/012:09:010
1054099130,10cyclictest0-21swapper/012:09:010
1055199129,2cyclictest7838-21grep12:00:153
1055199129,2cyclictest5914-21ssh09:10:363
1055199129,2cyclictest4332-21tail11:20:243
1055199129,2cyclictest429-21cut09:45:163
1055199129,2cyclictest30871-21head07:50:173
1055199129,2cyclictest27732-21ssh10:00:283
1055199129,2cyclictest25402-21grep07:40:133
1055199129,2cyclictest25138-21sed08:45:213
1055199129,2cyclictest23958-21cut10:55:133
1055199129,2cyclictest22057-21tail09:20:143
1055199129,2cyclictest13628-21tr07:15:153
1055199120,3cyclictest0-21swapper/311:07:083
1055199120,1cyclictest0-21swapper/310:07:273
1054699129,2cyclictest8225-21perf11:05:002
1054699129,2cyclictest6913-21cron08:09:592
1054699129,2cyclictest31480-21expr09:00:152
1054699129,2cyclictest24866-21chrt07:37:512
1054699129,2cyclictest23413-21cat10:35:272
1054699129,2cyclictest22212-21ssh11:12:302
1054699129,2cyclictest21163-21ssh10:53:242
1054699129,2cyclictest17000-21cut08:30:162
1054699129,2cyclictest1684-21grep09:45:262
1054699129,2cyclictest14064-21ssh12:04:132
1054699127,4cyclictest25079-21ntpq08:45:212
10546991210,1cyclictest8859-21grep08:10:252
10546991210,1cyclictest28819-21uname07:45:192
10546991210,1cyclictest15116-21head08:25:172
10546991210,1cyclictest11603-21grep10:10:172
1054699120,6cyclictest0-21swapper/209:40:102
1054699120,4cyclictest0-21swapper/209:10:132
1054699120,12cyclictest0-21swapper/210:46:162
1054699120,11cyclictest2141-21ssh12:35:002
1054399129,2cyclictest9433-21grep12:20:141
1054399129,2cyclictest9304-21awk08:15:001
1054399129,2cyclictest5505-21grep08:05:161
1054399129,2cyclictest2404-21grep12:15:271
1054399129,2cyclictest21727-21tr08:40:171
1054399129,2cyclictest2093-21head11:00:271
1054399129,2cyclictest18200-21tr07:25:141
1054399129,2cyclictest16459-21dump-pmu-power08:30:001
1054399129,2cyclictest13586-21tr07:15:151
1054399129,2cyclictest12735-21head08:20:171
1054399129,2cyclictest12735-21head08:20:171
1054399129,2cyclictest11275-21grep11:25:121
1054399128,3cyclictest18010-21ssh09:17:591
1054399128,3cyclictest10559-21cat11:25:001
1054399127,4cyclictest12091-21ntpq10:10:211
1054399122,6cyclictest30090-21apt-get07:50:131
1054099129,2cyclictest9987-21ssh10:09:490
1054099129,2cyclictest3559-21head12:35:180
1054099129,2cyclictest29992-21ssh10:02:000
1054099129,2cyclictest25040-21cron07:39:590
1054099129,2cyclictest2307-21ssh11:20:090
1054099129,2cyclictest20919-21ssh10:34:500
1054099129,2cyclictest20618-21ssh10:53:000
1054099129,2cyclictest15019-21grep07:15:260
1054099129,2cyclictest10600-21wc08:15:190
1054099129,2cyclictest10438-21ssh10:46:500
1054099123,5cyclictest3152-21diskmemload11:00:000
1054099121,9cyclictest14503-21cat09:34:590
1054099121,2cyclictest31977-21ntp_states11:55:200
1054099120,12cyclictest0-21swapper/012:19:320
1054099120,12cyclictest0-21swapper/011:34:260
1054099120,11cyclictest20425-21ssh10:15:180
1054099120,11cyclictest20425-21ssh10:15:170
1055199119,1cyclictest781-21grep07:55:163
1055199118,2cyclictest9299-21sh08:15:003
1055199118,2cyclictest9224-21cut12:20:133
1055199118,2cyclictest373-21ssh12:15:093
1055199118,2cyclictest30178-21ssh12:13:203
1055199118,2cyclictest29096-21ssh11:16:143
1055199118,2cyclictest2831-21grep12:35:153
1055199118,2cyclictest23259-21ssh12:28:033
1055199118,2cyclictest21625-21ssh10:35:123
1055199118,2cyclictest20238-21apt07:30:003
1055199118,2cyclictest1877-21cat09:05:183
1055199118,2cyclictest18558-21wc09:55:203
1055199118,2cyclictest18324-21sshd12:05:563
1055199118,2cyclictest18324-21sshd12:05:553
1055199118,2cyclictest18214-21ssh10:51:113
1055199118,2cyclictest15755-21ssh10:30:573
1055199118,2cyclictest14645-21ssh09:35:043
1055199118,2cyclictest12646-21grep09:15:153
1055199118,2cyclictest11954-21chrt08:20:113
1055199118,2cyclictest11954-21chrt08:20:113
1055199117,3cyclictest12736-21grep10:10:253
1055199116,3cyclictest21502-21aten2_r7power_c08:40:163
1055199112,6cyclictest21834-21apache211:25:093
1055199111,4cyclictest225-21systemd-journal08:40:003
1055199111,2cyclictest0-21swapper/312:30:243
1054699119,2cyclictest6548-21sh11:22:012
1054699119,2cyclictest3214-21ssh11:57:412
1054699119,2cyclictest16871-21sh09:17:082
1054699119,2cyclictest10303-21cat12:20:192
1054699119,1cyclictest30178-21rm10:40:062
1054699119,1cyclictest24078-21ssh12:10:042
1054699119,1cyclictest22181-21grep07:30:272
1054699119,1cyclictest19798-21grep07:25:262
1054699119,1cyclictest12898-21chrt07:12:472
1054699119,1cyclictest10460-21chrt08:15:172
1054699118,2cyclictest8447-21ssh09:30:382
1054699118,2cyclictest16121-21expr07:20:142
1054699118,2cyclictest1053-21grep09:05:142
1054699110,10cyclictest2863-21cat08:00:152
1054699110,10cyclictest12448-21df08:20:152
1054699110,10cyclictest12448-21df08:20:152
1054399118,2cyclictest653-21head09:45:171
1054399118,2cyclictest29815-21turbostat.cron07:49:591
1054399118,2cyclictest27442-21ssh09:23:331
1054399118,2cyclictest26885-21ssh10:56:091
1054399118,2cyclictest22170-21expr10:35:151
1054399118,2cyclictest20801-21ssh09:56:391
1054399118,2cyclictest13430-21sh12:03:441
1054399117,3cyclictest7695-21ssh10:45:181
1054399117,1cyclictest17849-21ssh11:47:221
1054399114,6cyclictest26137-21ssh10:00:181
1054399112,8cyclictest2171-21basename07:59:591
1054399110,1cyclictest0-21swapper/108:30:171
1054399110,0cyclictest0-21swapper/109:31:061
1054099119,2cyclictest7094-21sh09:11:290
1054099119,1cyclictest5315-21ssh11:40:160
1054099119,1cyclictest501-21ssh11:37:160
1054099119,1cyclictest3796-21awk08:00:230
1054099119,1cyclictest27958-21grep08:50:240
1054099119,1cyclictest23967-21cut08:45:130
1054099119,1cyclictest23651-21cron08:45:000
1054099119,1cyclictest22037-21tr10:35:150
1054099119,1cyclictest19720-21sed07:25:260
1054099119,1cyclictest139372sleep008:23:210
1054099119,1cyclictest139372sleep008:23:200
1054099118,2cyclictest5100-21tr08:05:140
1054099118,2cyclictest32742-21sed09:00:260
1054099118,2cyclictest31219-21cut10:40:180
1054099118,2cyclictest28611-21cat07:45:180
1054099118,2cyclictest26816-21ssh09:23:040
1054099118,2cyclictest21283-21sed07:30:180
1054099118,2cyclictest17130-21sh11:10:030
1054099118,2cyclictest14816-21ssh09:35:110
1054099118,2cyclictest1310-21fgrep09:45:230
1054099118,2cyclictest12843-21grep11:25:220
1054099118,2cyclictest10611-21ssh12:20:200
1054099115,2cyclictest225-21systemd-journal08:10:220
1054099113,7cyclictest27449-21munin-run07:45:000
1054099110,10cyclictest0-21swapper/010:29:240
1055199108,1cyclictest4873-21basename08:05:123
1055199108,1cyclictest129982sleep307:14:043
1055199107,2cyclictest3259-21idleruntime08:00:183
1055199106,3cyclictest31340-21tr09:00:143
1055199103,4cyclictest9568-21apt-get08:15:123
1054699109,1cyclictest700-21fschecks_time07:55:152
1054699108,2cyclictest3162-21expr12:35:162
1054699107,2cyclictest23505-21taskset08:43:122
1054699107,2cyclictest19293-21cat08:35:132
1054399108,1cyclictest21631-21ntpq07:30:221
1054399107,2cyclictest8179-21ssh09:50:011
1054399107,2cyclictest3810-21basename08:00:231
1054399107,2cyclictest26044-21apt08:50:011
1054399107,2cyclictest23353-21cat07:35:151
1054399107,2cyclictest1721-21grep09:05:161
1054399100,9cyclictest15885-21hostname07:20:131
1054099108,2cyclictest17175-21sort07:20:240
1054099108,1cyclictest30446-21chrt08:55:500
1054099108,1cyclictest21142-21chrt08:39:240
1054099108,1cyclictest16981-21tr08:30:160
1054099108,1cyclictest15687-21tr08:25:220
1054099102,2cyclictest225-21systemd-journal07:55:100
105519996,2cyclictest26930-21grep08:50:153
105469998,1cyclictest26299-21apt08:50:112
105439990,5cyclictest225-21systemd-journal07:15:001
105409997,1cyclictest29522sleep009:07:470
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional