You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-08 - 19:53

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot7s.osadl.org (updated Sat Aug 08, 2026 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2420729380,9sleep00-21swapper/007:07:060
2427329267,9sleep30-21swapper/307:07:583
2409729279,9sleep20-21swapper/207:05:402
2229828564,6sleep10-21swapper/107:05:061
98682720,0sleep00-21swapper/010:21:560
13562560,0sleep10-21swapper/109:57:491
2452999290,29cyclictest0-21swapper/212:26:422
2452399270,2cyclictest0-21swapper/011:29:350
2452399270,0cyclictest0-21swapper/011:53:080
2452699240,21cyclictest0-21swapper/112:30:141
24523992419,2cyclictest225-21systemd-journal11:36:390
2452999230,22cyclictest0-21swapper/211:53:082
2452999220,21cyclictest0-21swapper/209:30:082
2452999220,18cyclictest0-21swapper/210:40:022
2452399220,17cyclictest0-21swapper/009:38:420
2452999210,21cyclictest0-21swapper/210:02:292
2452999210,0cyclictest0-21swapper/211:29:352
24526992119,1cyclictest0-21swapper/111:55:191
24526992118,2cyclictest22102-21latency_hist10:29:591
20882210,1sleep30-21swapper/312:14:153
2452399207,7cyclictest3212-21grep08:39:590
24523992019,1cyclictest0-21swapper/009:33:030
24533991914,2cyclictest0-21swapper/311:48:183
24529991918,1cyclictest0-21swapper/209:41:032
2452999190,17cyclictest0-21swapper/209:36:152
24526991916,2cyclictest3344-21sort08:39:591
24523991912,2cyclictest0-21swapper/010:47:490
2453399187,9cyclictest0-21swapper/310:57:013
2452999182,15cyclictest0-21swapper/210:45:202
2452999180,13cyclictest26127-21sensors11:10:232
2452699183,13cyclictest17522-21diskmemload09:41:041
2452699182,15cyclictest17522-21diskmemload12:36:131
2452699180,18cyclictest0-21swapper/110:00:291
2452399180,6cyclictest225-21systemd-journal08:05:380
24533991716,1cyclictest0-21swapper/312:15:243
24529991716,1cyclictest0-21swapper/210:07:372
2452999171,11cyclictest0-21swapper/207:40:162
2452999170,2cyclictest0-21swapper/209:50:522
2452999170,2cyclictest0-21swapper/209:50:512
24526991713,3cyclictest25-21ksoftirqd/107:55:261
2452699170,17cyclictest0-21swapper/109:17:211
2452399172,13cyclictest12448-21ssh11:40:510
24523991716,1cyclictest13-21ksoftirqd/011:05:500
2452399170,16cyclictest0-21swapper/010:55:060
2452399170,16cyclictest0-21swapper/010:14:350
2453399162,14cyclictest0-21swapper/310:05:213
24533991614,1cyclictest0-21swapper/312:38:303
2453399160,2cyclictest0-21swapper/309:17:463
2452999160,3cyclictest0-21swapper/209:20:012
2452999160,15cyclictest0-21swapper/209:14:002
24526991614,1cyclictest25-21ksoftirqd/112:02:381
2452699160,3cyclictest0-21swapper/109:49:471
2452699160,1cyclictest0-21swapper/111:32:231
2452699160,0cyclictest0-21swapper/110:33:111
2452399160,3cyclictest0-21swapper/011:03:570
2452399160,12cyclictest17522-21diskmemload12:39:210
2452399160,11cyclictest0-21swapper/011:34:080
250462150,1sleep30-21swapper/310:11:383
2453399159,5cyclictest8894-21sh11:20:093
2453399155,5cyclictest17562-21cpuspeed_turbos11:25:153
2453399151,3cyclictest0-21swapper/312:05:173
2453399150,11cyclictest0-21swapper/308:40:123
2452999151,4cyclictest0-21swapper/210:28:192
2452999150,15cyclictest0-21swapper/212:10:262
2452999150,14cyclictest0-21swapper/211:16:232
2452999150,13cyclictest0-21swapper/210:16:312
2452699156,5cyclictest11944-21taskset10:23:341
24526991512,3cyclictest26262-21ssh10:12:351
2452699150,14cyclictest0-21swapper/109:32:361
2452699150,12cyclictest0-21swapper/111:40:111
2452399151,9cyclictest31127-21aten2_r7power_p10:35:170
24523991514,1cyclictest0-21swapper/011:21:290
2453399149,3cyclictest341-21grep08:30:263
24533991413,1cyclictest0-21swapper/311:41:453
24533991413,1cyclictest0-21swapper/310:35:003
24533991410,3cyclictest9603-21grep07:45:143
24533991410,3cyclictest13790-21cron10:25:003
24533991410,3cyclictest11163-21tr09:25:133
24533991410,2cyclictest32301-21cron11:15:003
2453399140,1cyclictest0-21swapper/309:48:563
24529991413,1cyclictest0-21swapper/207:30:152
2452999140,3cyclictest0-21swapper/209:56:212
2452699149,4cyclictest38150irq/131-snd_hda09:23:231
24526991412,1cyclictest18735-21cut11:45:151
2452399144,5cyclictest17522-21diskmemload11:10:160
2452399140,1cyclictest0-21swapper/010:53:220
2452399140,1cyclictest0-21swapper/009:18:210
2452399140,1cyclictest0-21swapper/008:20:120
2452399140,13cyclictest0-21swapper/011:15:200
2453399139,2cyclictest29633-21tr07:20:143
2453399139,2cyclictest1741-21cut08:35:183
24533991312,1cyclictest0-21swapper/311:03:113
24533991310,2cyclictest5554-21tail09:40:273
24533991310,2cyclictest39-21ksoftirqd/310:52:063
24533991310,2cyclictest20007-21tune2fs09:30:173
24533991310,2cyclictest18109-21grep08:00:273
24533991310,2cyclictest11913-21tail12:00:173
2452999135,5cyclictest477-21rs:main2
2452999133,9cyclictest20429-21/usr/sbin/munin12:05:192
24529991311,1cyclictest220172sleep211:08:272
24529991310,2cyclictest32-21ksoftirqd/210:21:002
24529991310,2cyclictest13989-21cat09:00:202
24529991310,2cyclictest13243-21grep07:50:252
24529991310,2cyclictest13243-21grep07:50:242
2452999130,9cyclictest0-21swapper/212:37:002
2452999130,13cyclictest0-21swapper/211:21:172
2452999130,0cyclictest0-21swapper/211:36:522
2452999130,0cyclictest0-21swapper/210:55:072
2452699139,4cyclictest9905-21kworker/u8:109:26:561
2452699139,2cyclictest10890-21tr08:55:161
2452699137,1cyclictest27609-21grep11:50:181
2452699135,7cyclictest20357-21apache_accesses09:50:131
2452699135,7cyclictest20357-21apache_accesses09:50:121
24526991310,2cyclictest9194-21cut07:45:131
24526991310,2cyclictest3089-21sendmail09:40:001
24526991310,2cyclictest2979-21tr07:30:221
24526991310,2cyclictest2012-21rm11:15:231
24526991310,2cyclictest12412-21head07:50:171
24526991310,2cyclictest12412-21head07:50:161
2452699130,3cyclictest0-21swapper/111:07:331
2452699130,0cyclictest0-21swapper/108:14:411
2452399137,2cyclictest225-21systemd-journal10:34:590
2452399134,7cyclictest0-21swapper/009:13:150
2452399130,1cyclictest0-21swapper/011:56:070
2453399129,2cyclictest6026-21ssh10:00:203
2453399129,2cyclictest471-21tail10:35:293
2453399129,2cyclictest3560-21grep11:55:203
2453399129,2cyclictest29402-21grep12:30:183
2453399129,2cyclictest29063-21ssh09:35:233
2453399129,2cyclictest28488-21wc11:50:273
2453399129,2cyclictest26700-21grep08:20:153
2453399129,2cyclictest21631-21tr08:10:173
2453399129,2cyclictest20454-21cut12:25:153
2453399129,2cyclictest13761-21grep09:00:173
2453399127,2cyclictest16139-21expr10:45:183
2453399122,6cyclictest2139-21wc11:15:233
2453399120,8cyclictest0-21swapper/309:55:323
2453399120,11cyclictest0-21swapper/308:50:143
2452999129,2cyclictest3988-21chrt11:55:232
2452999129,2cyclictest32461-21rm11:34:142
2452999129,2cyclictest25798-21taskset08:19:172
2452999129,2cyclictest24043-21cat10:50:152
2452999129,2cyclictest18386-21ssh11:45:122
2452999126,5cyclictest17522-21diskmemload10:10:422
2452999123,8cyclictest17522-21diskmemload10:30:162
24529991210,1cyclictest6383-21cat10:39:592
2452999120,9cyclictest0-21swapper/208:05:132
2452999120,2cyclictest0-21swapper/209:18:212
2452699129,2cyclictest8817-21cat07:45:001
2452699129,2cyclictest777-21cron08:34:591
2452699129,2cyclictest6328-21cron07:40:001
2452699129,2cyclictest30272-21tr07:20:171
2452699129,2cyclictest28193-21sort07:15:201
2452699129,2cyclictest2778-21cut11:35:221
2452699129,2cyclictest21966-21cat12:25:261
2452699129,2cyclictest20202-21grep12:05:181
2452699129,2cyclictest18133-21head11:25:171
2452699129,2cyclictest17399-21ls08:00:201
2452699129,2cyclictest15102-21grep10:05:271
2452699129,2cyclictest13884-21grep12:20:291
2452699128,3cyclictest426-21wc07:25:181
2452699127,4cyclictest27551-21ssh10:51:561
2452699123,7cyclictest25-21ksoftirqd/110:59:361
24526991210,1cyclictest4483-21sed12:15:191
2452399129,2cyclictest8803-21ssh10:02:010
2452399129,2cyclictest7855-21ssh09:42:110
2452399129,2cyclictest6897-21ssh10:40:120
2452399129,2cyclictest3756-21rm10:18:460
2452399129,2cyclictest20954-21sh10:29:090
2452399129,2cyclictest20800-21ls09:50:160
2452399129,2cyclictest20800-21ls09:50:150
2452399129,2cyclictest19694-21cut11:45:200
2452399129,2cyclictest1611-21sleep07:30:000
2452399129,2cyclictest12251-21tr12:20:160
2452399124,1cyclictest16144-21awk08:00:000
2452399124,1cyclictest11437-21awk12:20:000
2452399122,4cyclictest17522-21diskmemload12:25:110
24523991210,1cyclictest30723-21grep09:55:260
24523991210,1cyclictest28222-21tail07:15:200
2452399120,6cyclictest0-21swapper/009:21:310
233882120,1sleep30-21swapper/309:51:143
233882120,1sleep30-21swapper/309:51:143
2453399118,2cyclictest6901-21ssh11:38:063
2453399118,2cyclictest6638-21ssh10:40:013
2453399118,2cyclictest5175-21chrt09:20:553
2453399118,2cyclictest3909-21chrt07:34:293
2453399118,2cyclictest32519-21cut07:25:173
2453399118,2cyclictest30721-21apt-key08:30:003
2453399118,2cyclictest28364-21ssh11:31:013
2453399118,2cyclictest25479-21sed08:15:283
2453399118,2cyclictest19023-21ssh10:27:373
2453399118,2cyclictest18559-21tr09:10:153
2453399118,2cyclictest16067-21ssh11:05:073
2453399118,2cyclictest16046-21taskset07:59:133
2453399118,2cyclictest13374-21sed12:20:233
2453399118,2cyclictest12762-21tr07:50:203
2453399118,2cyclictest12762-21tr07:50:193
2453399118,2cyclictest1051-21ssh10:16:393
2452999119,2cyclictest8860-21cat11:00:182
2452999119,1cyclictest8937-21grep08:50:172
2452999119,1cyclictest7483-21ssh12:16:572
2452999119,1cyclictest5942-21aten2_r7power_v08:45:142
2452999119,1cyclictest4285-21aten2_r7power_e07:35:142
2452999119,1cyclictest30234-21grep08:25:272
2452999119,1cyclictest22633-21wc08:10:232
2452999119,1cyclictest16527-21ntpq09:05:202
2452999119,1cyclictest14441-21expr07:55:152
2452999119,1cyclictest13820-21sh11:41:562
2452999119,1cyclictest11682-21ssh12:20:032
2452999118,2cyclictest3803-21expr08:40:142
2452999118,2cyclictest31702-21cat07:25:002
2452999118,2cyclictest10562-21cat07:45:242
2452999113,5cyclictest17522-21diskmemload09:45:152
2452999111,2cyclictest2304-21sshd08:35:232
2452999111,2cyclictest11065-21apache_accesses12:00:122
2452999110,10cyclictest17580-21ssh09:29:072
2452699119,1cyclictest3726-21grep08:40:131
2452699119,1cyclictest25022-21wc08:15:251
2452699118,2cyclictest8796-21ssh11:00:171
2452699118,2cyclictest329-21wc10:35:281
2452699118,2cyclictest31590-21ssh11:14:261
2452699118,2cyclictest30437-21expr10:15:181
2452699118,2cyclictest20410-21grep08:05:261
2452699118,2cyclictest15156-21awk09:05:001
2452699118,2cyclictest10506-21wc11:20:241
2452699116,4cyclictest10075-21taskset08:51:201
2452699110,9cyclictest0-21swapper/110:45:131
2452699110,2cyclictest0-21swapper/109:10:501
2452399119,1cyclictest9492-21cat07:45:150
2452399119,1cyclictest6472sleep008:33:200
2452399119,1cyclictest5737-21cat08:45:010
2452399119,1cyclictest30217-21grep08:25:270
2452399119,1cyclictest27033-21sed07:15:000
2452399119,1cyclictest11604-21seq07:50:100
2452399119,1cyclictest11604-21seq07:50:090
2452399119,1cyclictest10862-21ssh12:00:040
2452399118,2cyclictest6651-21cut07:40:150
2452399118,2cyclictest3857-21ssh12:34:530
2452399118,2cyclictest29525-21apache_accesses07:20:130
2452399118,2cyclictest28049-21grep12:10:160
2452399118,2cyclictest23151-21chrt08:10:520
2452399118,2cyclictest18611-21sed08:05:000
2452399110,5cyclictest12386-21ssh09:25:200
2452399110,11cyclictest18058-21rm09:48:350
2452399110,10cyclictest13540-21sed09:00:160
2453399108,1cyclictest6559-21expr08:45:173
2453399108,1cyclictest27138-21dpkg07:15:143
2453399108,1cyclictest25063-21cstates07:10:143
2453399108,1cyclictest16966-21cat09:05:263
2453399107,2cyclictest6643-21cut07:40:163
2453399107,2cyclictest20801-21chrt08:08:593
2453399107,2cyclictest12087-21sort08:55:263
2453399102,6cyclictest4853-21grep07:35:173
2452999109,1cyclictest32465-21cut08:30:222
2452999109,1cyclictest18185-21taskset08:00:532
2452999109,1cyclictest11106-21cat08:55:162
2452999108,2cyclictest32031-21basename07:25:142
2452999107,2cyclictest25030-21grep07:10:142
2452999100,10cyclictest28253-21ntpq07:15:212
2452999100,0cyclictest0-21swapper/208:20:122
2452699108,1cyclictest7702-21taskset08:47:051
2452699108,1cyclictest7664-21tune2fs10:40:171
2452699108,1cyclictest28874-21expr08:25:161
2452699108,1cyclictest27471-21ssh12:10:031
2452699107,2cyclictest26635-21chrt07:12:151
2452699107,2cyclictest26160-21taskset08:20:131
2452399109,1cyclictest7597-21chrt08:45:420
2452399109,1cyclictest15099-21awk10:05:270
2452399108,1cyclictest5368-21ntpq07:35:200
2452399108,1cyclictest258322sleep008:19:430
2452399108,1cyclictest19148-21ssh12:05:020
2452399108,1cyclictest173462sleep009:07:590
2452399107,2cyclictest10157-21chrt08:52:230
2452399105,4cyclictest12739-21sh09:00:000
245269994,2cyclictest225-21systemd-journal09:05:011
245239998,1cyclictest3901-21chrt07:34:230
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional