You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-23 - 00:47

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot7s.osadl.org (updated Fri May 22, 2026 12:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1363329467,9sleep30-21swapper/307:06:413
1385329279,9sleep00-21swapper/007:09:310
1318928667,14sleep10-21swapper/107:05:261
1376628270,8sleep20-21swapper/207:08:252
74622620,0sleep30-21swapper/309:29:413
13985992927,1cyclictest11145-21kworker/3:211:50:023
1398399290,29cyclictest0-21swapper/210:00:252
1398399290,26cyclictest0-21swapper/209:35:192
1398399280,22cyclictest0-21swapper/210:36:512
1398399270,26cyclictest0-21swapper/209:20:572
13978992417,6cyclictest0-21swapper/009:42:050
1398599230,20cyclictest0-21swapper/309:45:133
1398399230,23cyclictest0-21swapper/212:13:542
1398399220,22cyclictest0-21swapper/211:53:352
1398399220,22cyclictest0-21swapper/210:24:522
1398399201,19cyclictest0-21swapper/210:59:182
1398099200,19cyclictest0-21swapper/110:38:431
13985991917,2cyclictest2455-21kworker/3:012:33:223
13985991916,2cyclictest11145-21kworker/3:211:29:123
13985991915,3cyclictest2455-21kworker/3:012:05:423
1398599190,14cyclictest7501-21apt-get09:10:133
1398599190,14cyclictest7501-21apt-get09:10:133
1398399190,19cyclictest0-21swapper/211:12:592
1398399190,19cyclictest0-21swapper/211:05:382
1398399190,17cyclictest0-21swapper/211:18:502
13980991917,2cyclictest0-21swapper/112:07:141
1397899192,3cyclictest141rcu_preempt11:39:170
1398099182,1cyclictest6995-21diskmemload10:55:571
13978991818,0cyclictest0-21swapper/010:37:260
1398399170,1cyclictest0-21swapper/210:40:232
1398399170,17cyclictest0-21swapper/212:28:522
1397899170,1cyclictest0-21swapper/011:45:490
13985991614,1cyclictest16689-21/usr/sbin/munin09:15:173
13983991613,0cyclictest0-21swapper/211:25:572
1398399161,1cyclictest822-21ssh12:19:332
13983991611,3cyclictest0-21swapper/209:54:482
1398399160,16cyclictest0-21swapper/211:02:362
1398399160,13cyclictest0-21swapper/209:17:302
1398099160,1cyclictest0-21swapper/112:00:591
1398099160,16cyclictest0-21swapper/112:36:191
1398099160,16cyclictest0-21swapper/111:02:521
1397899165,9cyclictest17491-21apache209:29:140
1397899160,2cyclictest0-21swapper/010:21:050
67002150,0sleep10-21swapper/112:22:341
67002150,0sleep10-21swapper/112:22:341
1398599151,2cyclictest141rcu_preempt10:35:083
1398599151,13cyclictest0-21swapper/310:40:153
1398399158,2cyclictest12483-21sort10:30:162
13983991514,1cyclictest0-21swapper/211:30:082
13983991511,1cyclictest0-21swapper/211:55:142
1398399150,1cyclictest0-21swapper/210:52:562
1398399150,14cyclictest0-21swapper/209:34:122
13980991513,1cyclictest0-21swapper/109:15:411
1398099151,13cyclictest0-21swapper/111:36:281
1398099150,15cyclictest0-21swapper/110:22:051
1398099150,14cyclictest0-21swapper/110:45:591
13978991515,0cyclictest0-21swapper/010:58:470
1397899150,13cyclictest0-21swapper/010:09:100
1398599143,2cyclictest141rcu_preempt11:06:593
13985991411,2cyclictest326-21grep07:45:263
1398599140,13cyclictest1250-21tr12:00:163
1398399142,2cyclictest7870-21munin-run09:30:012
1398399140,14cyclictest0-21swapper/211:41:172
1398399140,14cyclictest0-21swapper/211:21:112
13980991410,3cyclictest17348-21tail11:50:171
1398099140,1cyclictest0-21swapper/111:16:531
13978991410,3cyclictest11721-21head12:25:210
1397899140,14cyclictest0-21swapper/012:36:360
1397899140,13cyclictest3924-21grep07:55:140
1398599139,4cyclictest11145-21kworker/3:211:55:223
1398599139,3cyclictest31762-21grep11:20:163
1398599139,3cyclictest26373-21cat08:40:193
13985991312,1cyclictest0-21swapper/309:58:413
13985991311,1cyclictest0-21swapper/310:28:143
13985991310,2cyclictest6403-21sed10:45:263
13985991310,2cyclictest30873-21head08:50:173
13985991310,2cyclictest28243-21cat12:35:193
13985991310,2cyclictest18308-21grep08:25:143
1398599130,12cyclictest0-21swapper/312:20:153
1398599130,12cyclictest0-21swapper/312:20:143
1398399131,11cyclictest9127-21runrttasks12:24:292
1398399131,11cyclictest9127-21runrttasks12:24:292
13983991310,2cyclictest26391-21ntpq09:40:192
13983991310,2cyclictest26281-21grep07:35:162
13983991310,2cyclictest23909-21tr11:35:152
13983991310,2cyclictest22679-21wc07:25:202
13983991310,2cyclictest21114-21grep10:15:272
13983991310,2cyclictest17374-21grep07:15:182
13983991310,2cyclictest14979-21sed07:10:172
1398399130,12cyclictest0-21swapper/211:47:492
1398099139,3cyclictest3207-21sed10:05:161
1398099139,3cyclictest19879-21wc09:55:251
1398099139,3cyclictest18445-21cut12:10:171
1398099139,2cyclictest31261-21grep07:45:141
13980991311,1cyclictest0-21swapper/108:25:161
13980991310,2cyclictest8746-21grep11:25:251
13980991310,2cyclictest358-21tr08:55:141
13980991310,2cyclictest29006-21sed08:45:211
13980991310,2cyclictest25669-21grep09:40:141
13980991310,2cyclictest21199-21sed08:30:171
13980991310,2cyclictest17010-21tr09:35:141
13980991310,2cyclictest14194-21tr10:50:201
13980991310,2cyclictest10439-21head09:50:171
1398099130,7cyclictest0-21swapper/109:23:011
1398099130,12cyclictest9184-21tail08:05:181
1398099130,12cyclictest0-21swapper/112:25:011
1398099130,12cyclictest0-21swapper/111:06:041
1397899139,3cyclictest14416-21ls08:15:210
1397899135,5cyclictest1142-21runrttasks09:16:200
1397899131,6cyclictest7461-21sh11:45:000
13978991310,2cyclictest8253-21tr11:25:200
13978991310,2cyclictest19862-21sort10:15:170
13978991310,2cyclictest18190-21cat07:15:260
13978991310,2cyclictest17524-21cut09:35:170
13978991310,2cyclictest17321-21grep08:20:260
13978991310,2cyclictest12985-21basename10:50:130
1397899130,2cyclictest0-21swapper/009:52:590
1397899130,11cyclictest0-21swapper/012:21:160
1397899130,11cyclictest0-21swapper/012:21:160
1398599129,2cyclictest29195-21munin-plugin-st11:00:013
1398599129,2cyclictest28724-21uname08:45:193
1398599129,2cyclictest26551-21wget10:00:133
1398599129,2cyclictest26016-21cat09:20:243
1398599129,2cyclictest23588-21grep08:35:183
1398599129,2cyclictest19670-21cut07:20:153
1398599129,2cyclictest17625-21head09:35:183
1398599128,3cyclictest31676-21sed11:00:273
13985991211,1cyclictest0-21swapper/310:20:053
1398599120,11cyclictest0-21swapper/311:43:403
1398399129,2cyclictest8622-21wc09:10:182
1398399129,2cyclictest8622-21wc09:10:172
1398399129,2cyclictest6559-21sed09:05:262
1398399129,2cyclictest4480-21sed07:55:172
1398399129,2cyclictest30840-21sort08:50:162
1398399129,2cyclictest22128-21sed08:30:272
1398399129,2cyclictest15727-21tr08:20:132
1398399128,3cyclictest5390-21head10:45:162
1398399128,3cyclictest1549-21tr09:45:132
1398399128,3cyclictest12206-21gzip10:10:212
1398099129,2cyclictest7859-21cron09:30:001
1398099129,2cyclictest5380-21grep09:05:151
1398099129,2cyclictest31295-21ntpq08:50:201
1398099129,2cyclictest27006-21cron10:20:001
1398099129,2cyclictest19957-21cut12:30:191
1398099129,2cyclictest16094-21grep11:10:261
1398099129,2cyclictest15882-21sed07:10:261
1398099128,3cyclictest23523-21cron07:30:001
1398099127,4cyclictest14467-21ssh09:33:381
1398099123,5cyclictest31668-21grep11:20:161
1398099120,1cyclictest0-21swapper/111:31:521
1398099120,11cyclictest4728-21wc07:55:191
1397899129,2cyclictest5435-21df09:05:150
1397899129,2cyclictest5401-21sed10:45:170
1397899129,2cyclictest30301-21grep08:50:140
1397899129,2cyclictest24969-21grep09:20:150
1397899129,2cyclictest23915-21uname08:35:210
1397899129,2cyclictest2149-21grep12:00:210
1397899129,2cyclictest18231-21grep09:55:140
1397899128,3cyclictest3699-21grep10:25:130
1397899128,3cyclictest3246-21grep09:45:260
1397899120,2cyclictest0-21swapper/009:13:520
1397899120,2cyclictest0-21swapper/009:13:520
1397899120,1cyclictest0-21swapper/012:12:580
1397899120,11cyclictest0-21swapper/011:00:150
1398599119,1cyclictest843-21head08:55:163
1398599118,2cyclictest8760-21grep11:45:163
1398599118,2cyclictest8379-21cut08:05:003
1398599118,2cyclictest8099-21ssh09:30:013
1398599118,2cyclictest7128-21ssh10:07:363
1398599118,2cyclictest6525-21sed09:05:253
1398599118,2cyclictest31946-21ssh09:44:123
1398599118,2cyclictest3157-21chrt09:00:163
1398599118,2cyclictest29481-21ssh11:19:163
1398599118,2cyclictest26869-21tune2fs07:35:193
1398599118,2cyclictest25898-21cron07:35:003
1398599118,2cyclictest25881-21ssh12:15:093
1398599118,2cyclictest25015-21ls11:35:223
1398599118,2cyclictest23835-21ssh12:13:453
1398599118,2cyclictest22466-21ssh10:16:283
1398599118,2cyclictest22027-21expr08:30:263
1398599118,2cyclictest18762-21sh07:20:003
1398599118,2cyclictest16106-21ssh10:32:133
1398599118,2cyclictest15216-21rm10:12:223
1398599118,2cyclictest14916-21grep07:10:173
1398599118,2cyclictest14396-21tr11:10:143
1398599118,2cyclictest13662-21expr08:15:143
1398599118,2cyclictest11222-21cat08:10:163
1398599118,2cyclictest10367-21grep12:25:133
1398599110,10cyclictest18293-21rm11:31:143
1398599110,10cyclictest14483-21ssh10:50:223
1398399118,2cyclictest9043-21ssh10:28:142
1398399118,2cyclictest6375-21tr08:00:142
1398399118,2cyclictest32724-21egrep08:55:122
1398399118,2cyclictest3060-21grep09:00:152
1398399118,2cyclictest24288-21df07:30:152
1398399118,2cyclictest21340-21cat07:25:012
1398399118,2cyclictest2055-21ssh12:00:202
1398399118,2cyclictest19600-21head12:30:172
1398399118,2cyclictest13443-21grep08:15:142
1398399118,2cyclictest11648-21ssh12:05:492
1398399117,2cyclictest27186-21grep12:35:142
1398399111,6cyclictest1874-21sed07:50:172
1398399111,5cyclictest12944-21cron08:15:002
1398399110,11cyclictest0-21swapper/209:56:512
1398399110,10cyclictest9634-21ssh10:09:332
1398099119,1cyclictest906-21awk11:40:221
1398099118,2cyclictest7307-21sed08:00:211
1398099118,2cyclictest4476-21uname10:25:171
1398099118,2cyclictest3228-21sed09:45:261
1398099118,2cyclictest30551-21grep10:40:251
1398099118,2cyclictest29545-21sed07:40:161
1398099118,2cyclictest25974-21ssh11:55:191
1398099118,2cyclictest17750-21grep07:15:201
1398099118,2cyclictest15857-21grep08:20:151
1398099116,4cyclictest4577-21taskset09:04:341
1398099113,5cyclictest11239-21/usr/sbin/munin10:10:141
1398099112,3cyclictest3231-21munin-run07:55:001
1398099110,11cyclictest0-21swapper/111:48:361
1397899118,2cyclictest9957-21grep12:05:170
1397899118,2cyclictest8902-21grep09:30:150
1397899118,2cyclictest7479-21chrt08:00:230
1397899118,2cyclictest3633-21ssh10:44:410
1397899118,2cyclictest31016-21chrt12:17:490
1397899118,2cyclictest30900-21ssh11:20:120
1397899118,2cyclictest30002-21grep07:40:210
1397899118,2cyclictest27678-21ssh11:17:520
1397899118,2cyclictest27592-21wc08:45:000
1397899118,2cyclictest26590-21grep10:00:130
1397899118,2cyclictest25839-21chrt07:34:140
1397899118,2cyclictest25008-21ssh11:55:130
1397899118,2cyclictest17719-21ssh11:11:380
1397899118,2cyclictest16393-21cut11:30:190
1397899118,2cyclictest15152-21ssh10:12:190
1397899118,2cyclictest13319-21wc10:30:220
1397899117,4cyclictest1035-21packagekitd08:07:390
1397899117,3cyclictest21055-21sort08:30:170
1397899112,5cyclictest3227-21munin-plugin-st07:55:000
1398599108,1cyclictest28585-21sed07:40:013
1398599108,1cyclictest22309-21cat07:25:163
1398599108,1cyclictest2119-21cat07:50:193
1398599107,2cyclictest9483-21ssh09:50:073
1398599107,2cyclictest3571-21gpg-connect-age07:55:003
1398599107,2cyclictest16301-21uniq08:20:173
1398599100,9cyclictest9966-21cat08:05:283
1398399108,1cyclictest9161-21grep08:05:182
1398399108,1cyclictest29990-21date08:50:002
1398399108,1cyclictest18441-21expr08:25:142
1398399107,2cyclictest25680-21grep08:40:152
1398399107,2cyclictest24564-21chrt08:35:392
1398399106,3cyclictest32761-21grep07:45:262
1398099108,1cyclictest7250-21ssh09:10:021
1398099108,1cyclictest7250-21ssh09:10:021
1398099108,1cyclictest27022-21sed10:00:141
1398099108,1cyclictest246882sleep108:37:141
1398099108,1cyclictest14540-21grep08:15:211
1398099107,2cyclictest28243-21taskset07:39:391
1398099107,2cyclictest27078-21tail08:40:261
1398099107,2cyclictest24906-21cut07:30:191
1398099107,2cyclictest19119-21grep07:20:131
1398099107,2cyclictest13738-21grep10:30:261
1398099107,2cyclictest12412-21wc08:10:261
1398099100,9cyclictest26227-21wc12:15:141
1397899108,1cyclictest28024-21taskset07:36:500
1397899108,1cyclictest18564-21ssh12:30:060
1397899108,1cyclictest18124-21wget08:25:130
1397899108,1cyclictest11916-21sed08:10:190
1397899107,2cyclictest823-21cron07:50:010
1397899107,2cyclictest32747-21cat08:55:140
1397899107,2cyclictest3205-21tune2fs09:00:160
1397899107,2cyclictest30043-21grep08:50:000
1397899107,2cyclictest19693-21grep07:20:160
1397899107,2cyclictest16258-21ssh11:50:010
1397899100,9cyclictest6183-21expr11:05:150
139839997,1cyclictest28988-21cat07:40:142
139789997,1cyclictest21723-21cat07:25:130
139789990,5cyclictest0-21swapper/007:10:130
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional