You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-06 - 22:53

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Fri Mar 06, 2026 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2456528774,9sleep20-21swapper/207:09:412
2439528371,8sleep00-21swapper/007:07:310
2436327967,8sleep30-21swapper/307:07:053
2245927857,6sleep10-21swapper/107:05:061
283092550,0sleep00-21swapper/009:15:290
2467799320,2cyclictest17478-21diskmemload10:44:590
2467799300,7cyclictest1887-21idleruntime-cro12:14:590
2467799290,5cyclictest16380-21ls10:45:250
24684992322,1cyclictest0-21swapper/311:56:513
2468299230,21cyclictest0-21swapper/212:35:412
2467799230,1cyclictest0-21swapper/011:12:070
2468299220,21cyclictest0-21swapper/209:51:232
2467799222,5cyclictest1142-21runrttasks09:48:000
2468299211,20cyclictest0-21swapper/210:32:502
2468299210,20cyclictest0-21swapper/212:27:042
2468299210,18cyclictest0-21swapper/211:57:162
2467799216,1cyclictest0-21swapper/012:35:080
2467799216,11cyclictest24668-21cyclictest11:05:380
2467799214,2cyclictest17478-21diskmemload10:15:110
2467799213,1cyclictest17478-21diskmemload10:01:240
2467799210,20cyclictest0-21swapper/011:33:350
24678992019,1cyclictest0-21swapper/110:45:331
24678992013,3cyclictest1206-21chrt12:14:291
170372200,0sleep20-21swapper/210:07:272
24684991919,0cyclictest0-21swapper/311:04:243
2468299192,13cyclictest17820-21/usr/sbin/munin11:25:242
24682991919,0cyclictest0-21swapper/209:21:032
2468299190,19cyclictest0-21swapper/211:49:332
2467899190,3cyclictest0-21swapper/111:56:101
2467799190,16cyclictest0-21swapper/008:45:140
2467799190,12cyclictest0-21swapper/011:17:480
2467799190,11cyclictest0-21swapper/011:22:550
2468499180,2cyclictest0-21swapper/311:30:223
2468299180,18cyclictest0-21swapper/210:24:222
2467899183,6cyclictest13039-21apt-config10:25:001
2467899183,2cyclictest141rcu_preempt09:48:271
2467799187,6cyclictest27033-21awk07:15:000
2467799181,6cyclictest1142-21runrttasks11:54:290
2468499170,1cyclictest0-21swapper/312:08:143
24682991717,0cyclictest0-21swapper/210:51:162
2467899179,2cyclictest0-21swapper/109:31:061
24678991717,0cyclictest0-21swapper/111:52:461
24678991714,2cyclictest0-21swapper/110:02:591
24678991712,2cyclictest4196-21apt-key07:34:591
2467799178,2cyclictest11970-21grep07:50:140
2467799173,2cyclictest0-21swapper/009:24:470
24677991715,2cyclictest13-21ksoftirqd/011:03:250
24677991710,6cyclictest635-21grep10:55:250
24677991710,6cyclictest13073-21ssh10:25:010
24677991710,6cyclictest12956-21grep09:00:150
24677991710,5cyclictest22825-21tail10:30:170
2468499168,5cyclictest537-21snmpd10:57:413
2468499160,1cyclictest0-21swapper/311:16:153
2468499160,14cyclictest0-21swapper/311:43:243
2468499160,0cyclictest0-21swapper/312:27:083
2468299160,16cyclictest0-21swapper/212:13:382
24678991615,1cyclictest0-21swapper/109:40:471
24678991610,4cyclictest25153-21tr07:10:141
2467899160,9cyclictest0-21swapper/111:26:041
2467899160,15cyclictest0-21swapper/112:22:401
2467799169,6cyclictest28070-21cut12:30:170
2467799167,2cyclictest16835-21ssh12:04:340
2467799160,4cyclictest0-21swapper/011:44:340
2467799160,2cyclictest0-21swapper/008:30:140
24684991514,1cyclictest0-21swapper/312:34:183
2468499150,3cyclictest0-21swapper/309:46:463
2468499150,14cyclictest0-21swapper/309:29:373
2468499150,11cyclictest0-21swapper/309:00:143
2468299151,13cyclictest29236-21ssh09:55:182
2468299150,15cyclictest0-21swapper/211:42:442
2468299150,14cyclictest0-21swapper/212:01:052
2467899156,8cyclictest19078-21munin-node10:18:001
2467899156,5cyclictest225-21systemd-journal11:17:001
2467899155,9cyclictest38150irq/131-snd_hda11:12:241
2467899152,8cyclictest0-21swapper/111:05:261
2467899152,12cyclictest0-21swapper/112:39:001
2467899151,9cyclictest17499-21ssh10:26:561
24678991511,2cyclictest38150irq/131-snd_hda11:21:421
2467899151,10cyclictest0-21swapper/109:15:011
2467799159,5cyclictest2749-21grep07:30:170
2467799159,5cyclictest16457-21grep09:05:200
2467799158,2cyclictest26949-21grep08:20:160
2467799158,2cyclictest10713-21cut09:25:150
2467799158,1cyclictest5535-21sleep08:45:000
2467799158,1cyclictest28939-21cat09:35:260
2467799158,1cyclictest11477-21grep08:55:180
2467799157,2cyclictest2727-21sed08:35:260
2467799157,2cyclictest21590-21ssh12:25:500
2467799155,6cyclictest12422-21apt-get10:05:130
2467799153,7cyclictest16306-21awk07:59:590
24677991513,1cyclictest17478-21diskmemload10:11:290
2467799150,7cyclictest0-21swapper/011:27:050
2467799150,1cyclictest0-21swapper/011:58:170
2468499142,8cyclictest537-21snmpd12:13:443
2468499142,7cyclictest0-21swapper/308:55:003
2468499141,9cyclictest27108-21/usr/sbin/munin09:35:193
24684991411,2cyclictest4874-21chrt09:20:573
24682991412,1cyclictest17478-21diskmemload09:36:572
24682991410,3cyclictest11532-21ssh09:25:192
2468299140,3cyclictest0-21swapper/210:25:482
2467899141,7cyclictest12292-21switchtime09:25:261
24678991410,3cyclictest26392-21sed11:30:261
2467799148,1cyclictest90992sleep010:22:040
2467799148,1cyclictest842-21sh10:36:130
2467799148,1cyclictest212222sleep009:30:380
2467799147,2cyclictest4634-21grep07:35:150
2467799147,2cyclictest31041-21ssh09:56:090
2467799147,1cyclictest19629-21taskset11:45:520
2467799146,2cyclictest20513-21tail09:50:150
2467799142,10cyclictest19479-21sensors12:05:240
2467799141,3cyclictest225-21systemd-journal10:55:000
24677991410,3cyclictest25312-21sed08:15:250
2468499139,3cyclictest5332-21cut10:20:153
2468499139,3cyclictest19623-21sort08:05:163
2468499139,3cyclictest14827-21grep10:45:143
24684991311,2cyclictest0-21swapper/309:53:093
24684991310,2cyclictest4128-21grep10:00:123
24684991310,2cyclictest31430-21cron11:15:013
24684991310,2cyclictest139862sleep310:25:153
2468499130,10cyclictest0-21swapper/311:50:563
2468299139,3cyclictest26959-21sort08:20:172
2468299139,3cyclictest17280-21grep11:05:262
2468299133,9cyclictest6993-21/usr/sbin/munin11:00:282
24682991310,3cyclictest55650irq/127-enp2s0-09:44:042
24682991310,2cyclictest8408-21grep08:50:132
24682991310,2cyclictest4856-21grep10:00:162
24682991310,2cyclictest375-21sed07:25:152
24682991310,2cyclictest28915-21grep08:25:132
24682991310,2cyclictest19875-21grep09:10:272
24682991310,2cyclictest19608-21cat12:05:252
24682991310,2cyclictest15863-21grep07:55:252
2468299130,2cyclictest0-21swapper/210:36:332
2468299130,1cyclictest0-21swapper/210:11:052
2468299130,12cyclictest32619-21sh10:16:372
2468299130,12cyclictest0-21swapper/211:18:092
2467899139,3cyclictest8185-21dump-pmu-power11:40:001
2467899132,6cyclictest17478-21diskmemload11:47:461
24678991310,2cyclictest31303-21grep08:30:151
24678991310,2cyclictest21060-21grep10:10:151
24678991310,2cyclictest13693-21head09:00:181
2467899130,2cyclictest0-21swapper/112:00:141
2467899130,1cyclictest17478-21diskmemload09:15:061
2467799139,3cyclictest7239-21tr07:40:130
2467799139,2cyclictest30428-21tail07:20:240
2467799138,1cyclictest1183-21grep11:35:150
2467799138,1cyclictest10788-21ssh09:44:550
2467799137,1cyclictest291852sleep007:17:280
2467799135,2cyclictest10985-21tr12:20:140
24677991310,2cyclictest3118-21sort12:15:160
24677991310,2cyclictest28945-21grep08:25:140
2467799130,10cyclictest23284-21taskset08:13:210
2468499129,2cyclictest8080-21grep07:40:183
2468499129,2cyclictest29368-21grep10:15:133
2468499129,2cyclictest27209-21cut09:15:183
2468499129,2cyclictest23434-21basename08:15:013
2468499129,2cyclictest20954-21head09:30:273
2468499129,2cyclictest12614-21grep12:20:253
2468499128,3cyclictest4169-21grep08:40:153
2468499128,3cyclictest19922-21sed09:10:273
2468499124,4cyclictest225-21systemd-journal11:10:003
2468499120,6cyclictest0-21swapper/310:50:013
2468499120,12cyclictest0-21swapper/312:39:443
2468299129,2cyclictest7023-21grep10:40:152
2468299129,2cyclictest30426-21grep07:20:242
2468299129,2cyclictest27037-21grep09:15:182
2468299129,2cyclictest23939-21grep11:10:142
2468299129,2cyclictest17050-21grep09:05:272
2468299129,2cyclictest11750-21grep08:55:202
2468299123,2cyclictest14279-21diskmemload12:21:392
24682991210,1cyclictest31989-21tail10:55:162
24682991210,1cyclictest16808-21grep08:00:152
24682991210,1cyclictest15887-21ls10:45:202
2467899129,2cyclictest8253-21grep08:50:131
2467899129,2cyclictest7037-21wc08:45:231
2467899129,2cyclictest5360-21grep07:35:181
2467899129,2cyclictest29404-21ls07:20:001
2467899129,2cyclictest18206-21grep08:00:271
2467899129,2cyclictest16679-21wc09:05:231
2467899128,3cyclictest453-21cat10:55:251
2467899128,2cyclictest28275-21wc09:35:191
2467899127,4cyclictest5202-21taskset08:41:101
2467899127,4cyclictest2894-21ssh09:59:291
2467899124,4cyclictest273-21systemd-udevd10:08:031
2467899123,1cyclictest20879-21cat09:50:181
2467899120,11cyclictest12678-21sleep09:00:001
2467799129,2cyclictest20635-21grep08:05:250
2467799129,2cyclictest10084-21tail07:45:160
2467799127,1cyclictest10194-21taskset08:54:200
2468499119,1cyclictest5955-21taskset10:40:023
2468499119,1cyclictest17911-21grep11:45:173
2468499118,2cyclictest9315-21ssh12:00:043
2468499118,2cyclictest6783-21ssh09:41:473
2468499118,2cyclictest6736-21ssh12:17:163
2468499118,2cyclictest5357-21head07:35:173
2468499118,2cyclictest4499-21ssh11:37:073
2468499118,2cyclictest31346-21tr08:30:163
2468499118,2cyclictest31083-21ssh09:56:123
2468499118,2cyclictest3087-21ssh10:37:593
2468499118,2cyclictest29213-21ssh10:34:353
2468499118,2cyclictest27924-21sed08:20:263
2468499118,2cyclictest25996-21grep07:10:203
2468499118,2cyclictest21099-21cut10:10:153
2468499118,2cyclictest18243-21ssh11:25:273
2468499118,2cyclictest17053-21sed09:05:273
2468499118,2cyclictest14887-21grep07:55:173
2468499118,2cyclictest14673-21ssh10:05:373
2468499118,2cyclictest1316-21grep07:25:253
2468499118,2cyclictest12020-21tr07:50:143
2468499118,2cyclictest11363-21cat08:55:183
2468499110,10cyclictest25354-21sed08:15:253
2468299118,2cyclictest4077-21sort08:40:152
2468299118,2cyclictest3182-21grep12:15:172
2468299118,2cyclictest30021-21sh12:30:562
2468299118,2cyclictest26135-21rm11:50:172
2468299118,2cyclictest25836-21readlink08:20:002
2468299118,2cyclictest25625-21grep07:10:172
2468299118,2cyclictest21576-21tr08:10:142
2468299118,2cyclictest21011-21taskset08:09:372
2468299118,2cyclictest19748-21tune2fs09:30:172
2468299118,2cyclictest15055-21taskset09:04:432
2468299118,2cyclictest12265-21cut09:45:172
2468299118,2cyclictest12208-21cat07:50:142
2468299118,2cyclictest11099-21ssh11:21:242
24682991110,1cyclictest0-21swapper/211:30:072
2468299110,11cyclictest0-21swapper/207:45:132
2467899118,2cyclictest9900-21ssh12:19:471
2467899118,2cyclictest8586-21ssh11:40:081
2467899118,2cyclictest7364-21cat10:40:181
2467899118,2cyclictest4197-21head09:20:251
2467899118,2cyclictest30706-21ssh12:31:291
2467899118,2cyclictest25130-21sshd10:50:411
2467899118,2cyclictest24197-21sh12:08:481
2467899118,2cyclictest22533-21ls08:10:211
2467899118,2cyclictest21999-21aten2_r7power_c10:30:141
2467899118,2cyclictest21497-21ssh12:25:451
2467899118,2cyclictest12002-21tr07:50:141
2467899118,2cyclictest1104-21grep08:35:141
2467899117,2cyclictest0-21swapper/107:25:121
2467799118,2cyclictest719-21cat07:25:200
2468499108,1cyclictest9581-21cpuspeed_turbos07:45:153
2468499108,1cyclictest7240-21sed08:45:273
2468499108,1cyclictest27672-21ls07:15:143
2468499108,1cyclictest2703-21grep07:30:173
2468499108,1cyclictest21652sleep308:35:203
2468499107,2cyclictest30556-21taskset08:27:073
2468499107,2cyclictest29968-21expr07:20:233
2468499107,2cyclictest18241-21tail08:00:263
2468499104,3cyclictest0-21swapper/311:20:153
2468299108,1cyclictest1962-21rm11:35:222
2468299108,1cyclictest1326-21expr08:35:152
2468299107,2cyclictest7566-21df07:40:142
2468299107,2cyclictest6002-21tr08:45:162
2468299107,2cyclictest4129-21apt07:34:592
2468299107,2cyclictest28593-21gzip07:15:212
2468299101,6cyclictest0-21swapper/208:30:142
2468299100,8cyclictest0-21swapper/207:35:142
2467899108,1cyclictest7303-21cpuspeed_turbos11:00:151
2467899108,1cyclictest7085-21cut07:40:121
2467899108,1cyclictest31053-21mailstats07:20:291
2467899108,1cyclictest30163-21ssh10:35:091
2467899108,1cyclictest28881-21grep08:25:131
2467899108,1cyclictest28084-21taskset08:20:501
2467899108,1cyclictest160512sleep107:57:111
2467899107,2cyclictest23784-21cut08:15:131
2467899107,2cyclictest18904-21taskset08:05:041
2467899107,2cyclictest10027-21grep07:45:171
2467799107,2cyclictest18011-21ssh09:10:130
2467799107,2cyclictest17539-21sed08:00:200
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional