You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-01 - 21:56

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot7s.osadl.org (updated Fri May 01, 2026 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
10004210464,36sleep00-21swapper/007:06:180
1017629380,9sleep20-21swapper/207:08:322
1011429266,9sleep30-21swapper/307:07:453
1000028564,6sleep10-21swapper/107:06:141
234502710,0sleep20-21swapper/209:40:342
1038699380,37cyclictest0-21swapper/210:55:222
1038699270,1cyclictest0-21swapper/210:35:522
10386992320,3cyclictest0-21swapper/210:20:422
1038699230,1cyclictest3183-21diskmemload10:52:032
1038399230,23cyclictest0-21swapper/110:43:571
1038199230,1cyclictest0-21swapper/009:34:040
10388992218,1cyclictest39-21ksoftirqd/310:08:133
10388992217,4cyclictest0-21swapper/310:13:493
1038699220,22cyclictest0-21swapper/211:19:022
1038699220,22cyclictest0-21swapper/210:29:032
1038399220,22cyclictest0-21swapper/109:53:491
10388992120,1cyclictest0-21swapper/310:38:453
10388992118,1cyclictest0-21swapper/310:41:183
1038899210,3cyclictest0-21swapper/311:41:093
10381992118,2cyclictest13-21ksoftirqd/010:50:190
10388992017,1cyclictest0-21swapper/310:30:203
10388992017,1cyclictest0-21swapper/310:26:293
10388992017,1cyclictest0-21swapper/310:22:303
1038699200,20cyclictest0-21swapper/211:23:032
1038399200,2cyclictest0-21swapper/109:13:501
1038399200,20cyclictest0-21swapper/112:25:191
10388991916,2cyclictest0-21swapper/310:48:063
1038699190,19cyclictest0-21swapper/210:43:422
1038399198,10cyclictest13011-21munin-node09:31:031
10383991919,0cyclictest0-21swapper/109:26:001
1038699180,17cyclictest0-21swapper/210:05:212
1038399185,12cyclictest20451-21sendmail_mailtr11:15:231
1038399180,18cyclictest0-21swapper/112:23:111
194792170,0sleep20-21swapper/212:32:172
10388991716,1cyclictest0-21swapper/309:00:143
10388991715,2cyclictest0-21swapper/310:19:343
1038899171,2cyclictest0-21swapper/311:03:583
10388991712,4cyclictest39-21ksoftirqd/309:21:403
1038899170,1cyclictest0-21swapper/311:16:333
1038399179,6cyclictest23986-21basename08:45:141
10383991716,1cyclictest0-21swapper/111:21:121
1038399170,14cyclictest0-21swapper/107:20:111
1038199170,2cyclictest0-21swapper/012:38:030
216672160,2chrt225-21systemd-journal09:59:590
10388991614,2cyclictest39-21ksoftirqd/312:04:453
10388991614,1cyclictest39-21ksoftirqd/311:51:523
1038899160,1cyclictest0-21swapper/311:30:043
1038699163,1cyclictest141rcu_preempt12:26:152
10383991615,1cyclictest0-21swapper/110:50:461
1038399160,15cyclictest0-21swapper/112:06:301
1038199165,6cyclictest6249-21munin-run12:25:000
1038199160,1cyclictest141rcu_preempt09:22:550
1038199160,11cyclictest29383-21fw_packets09:25:160
1038899158,5cyclictest39-21ksoftirqd/312:05:213
1038899157,7cyclictest31297-21latency_hist12:39:593
10388991514,1cyclictest391ksoftirqd/309:41:353
10388991511,3cyclictest39-21ksoftirqd/309:48:073
10388991511,3cyclictest22220-21ls11:55:183
10383991514,1cyclictest25-21ksoftirqd/109:40:041
10383991514,1cyclictest0-21swapper/110:33:131
10383991512,2cyclictest3183-21diskmemload10:37:221
1038399150,2cyclictest0-21swapper/110:08:341
1038399150,1cyclictest0-21swapper/111:08:381
1038399150,0cyclictest0-21swapper/110:49:521
1038199150,1cyclictest141rcu_preempt10:31:030
1038199150,1cyclictest0-21swapper/012:34:200
43432143,1sleep03183-21diskmemload10:08:340
1038899144,5cyclictest4106-21cron09:30:003
1038899141,8cyclictest4108-21kworker/u8:111:45:123
1038899141,10cyclictest0-21swapper/309:33:413
10388991410,3cyclictest39-21ksoftirqd/309:10:243
1038899140,14cyclictest0-21swapper/312:23:233
10386991411,2cyclictest24131-21grep07:35:252
10386991410,2cyclictest14229-21tr07:15:222
10383991412,2cyclictest0-21swapper/110:19:101
10383991412,2cyclictest0-21swapper/109:19:281
10383991411,3cyclictest0-21swapper/111:29:461
1038199143,6cyclictest31791-21ssh10:25:140
1038199141,7cyclictest161rcuc/009:45:260
10381991414,0cyclictest0-21swapper/011:01:540
10381991412,1cyclictest13-21ksoftirqd/010:40:200
10381991411,2cyclictest29294-21tune2fs08:55:150
10381991411,2cyclictest25329-21grep10:20:260
1038199140,3cyclictest0-21swapper/007:10:130
127902130,2sleep20-21swapper/212:09:392
1038899136,5cyclictest0-21swapper/311:20:143
1038899134,8cyclictest0-21swapper/309:05:133
10388991311,1cyclictest39-21ksoftirqd/309:15:203
10388991310,2cyclictest5546-21grep09:50:113
10388991310,2cyclictest39-21ksoftirqd/307:15:253
10388991310,2cyclictest3414-21wc11:05:213
10388991310,2cyclictest28879-21cut08:55:133
10388991310,2cyclictest21052-21systemctl08:40:013
10388991310,2cyclictest18380-21cut07:25:163
10388991310,2cyclictest10838-21grep11:10:143
10388991310,2cyclictest0-21swapper/312:15:093
1038699139,3cyclictest30522-21wc12:00:182
1038699137,5cyclictest11730-21cat11:10:192
1038699132,4cyclictest3183-21diskmemload12:39:592
10386991310,2cyclictest4293-21wc09:30:012
10386991310,2cyclictest32150-21wc12:20:212
10386991310,2cyclictest21341-21tr11:55:132
1038399139,3cyclictest15697-21ssh09:36:031
1038399138,2cyclictest16895-21ssh11:33:001
1038399131,4cyclictest225-21systemd-journal11:55:001
10383991310,2cyclictest27517-21tr07:45:141
10383991310,2cyclictest21615-21cat07:30:251
10383991310,2cyclictest12378-21grep11:10:241
1038399130,10cyclictest0-21swapper/110:22:351
1038199139,3cyclictest21075-21sleep08:40:000
1038199139,2cyclictest8426-21wc10:10:250
1038199138,3cyclictest12179-21cron11:50:010
1038199135,7cyclictest18682-21awk08:35:000
1038199131,12cyclictest0-21swapper/011:29:130
10381991310,2cyclictest16067-21cron08:30:010
1038899129,2cyclictest5063-21wc11:25:243
1038899129,2cyclictest25429-21cron10:59:593
1038899129,2cyclictest24346-21head10:00:273
1038899129,2cyclictest17306-21sed08:30:153
1038899129,2cyclictest14086-21tr09:55:123
1038899129,2cyclictest12560-21chrt10:51:293
1038899128,3cyclictest4693-21tr08:05:153
1038899128,3cyclictest27924-21cut07:45:173
1038899128,3cyclictest20372-21tr07:30:143
1038899126,2cyclictest225-21systemd-journal08:15:003
1038899120,11cyclictest24137-21cut08:45:153
1038699129,2cyclictest9390-21ssh10:30:192
1038699129,2cyclictest5551-21sed09:10:252
1038699129,2cyclictest5161-21grep11:25:252
1038699129,2cyclictest26992-21grep08:50:162
1038699129,2cyclictest23158-21grep10:00:162
1038699129,2cyclictest22919-21sleep10:20:002
1038699129,2cyclictest21815-21grep11:35:252
1038699129,2cyclictest21815-21grep11:35:242
1038699129,2cyclictest19412-21sed07:25:262
1038699129,2cyclictest14939-21ssh09:55:172
1038699129,2cyclictest14671-21grep08:25:172
1038699128,3cyclictest11796-21tr11:30:112
1038699128,2cyclictest20884-21sed07:30:162
1038699128,2cyclictest2007-21grep11:05:132
1038699127,3cyclictest29374-21head08:55:152
1038699124,4cyclictest225-21systemd-journal09:55:002
1038699121,6cyclictest225-21systemd-journal07:25:002
1038699121,2cyclictest1102-21timerandwakeup07:55:242
1038699120,12cyclictest0-21swapper/212:13:372
1038699120,11cyclictest26791-21head11:00:182
1038699120,11cyclictest12214-21grep08:20:162
1038399129,2cyclictest5160-21tail11:45:171
1038399129,2cyclictest32658-21grep09:00:271
1038399129,2cyclictest3183-21diskmemload12:19:591
1038399129,2cyclictest29871-21grep08:55:201
1038399129,2cyclictest15008-21grep09:55:171
1038399129,2cyclictest14325-21df12:10:141
1038399128,3cyclictest27027-21grep08:50:171
1038399128,1cyclictest22850-21ssh11:36:081
1038399128,1cyclictest22850-21ssh11:36:071
1038399122,8cyclictest13453-1kworker/u9:211:00:141
10383991212,0cyclictest0-21swapper/109:20:301
10383991210,1cyclictest12549-21cat08:20:181
1038199129,2cyclictest6179-21cut12:05:170
1038199129,2cyclictest30196-21grep11:40:250
1038199129,2cyclictest27948-21grep08:50:260
1038199129,2cyclictest2706-21sed09:05:260
1038199129,2cyclictest23451-21ssh09:40:340
1038199129,2cyclictest23134-21grep12:15:150
1038199129,2cyclictest22930-21grep08:40:260
1038199129,2cyclictest21824-21grep07:30:250
1038199129,2cyclictest13418-21cut11:50:150
1038199129,2cyclictest13311-21cron09:55:000
1038199129,2cyclictest13-21ksoftirqd/012:10:130
1038199128,3cyclictest853-21wc07:55:200
1038199128,2cyclictest20957-21cat09:39:590
10381991210,1cyclictest3501-21ssh12:03:530
1038199120,1cyclictest0-21swapper/010:00:460
1038199120,12cyclictest0-21swapper/011:38:490
1038199120,12cyclictest0-21swapper/011:38:480
1038199120,12cyclictest0-21swapper/010:46:390
1038199120,11cyclictest18924-21chrt11:15:140
1038899119,1cyclictest23704-21cut08:45:003
1038899119,1cyclictest118672sleep307:10:223
1038899118,2cyclictest7119-21cat12:25:153
1038899118,2cyclictest32741-21grep07:55:143
1038899118,2cyclictest26498-21tr08:50:143
1038899118,2cyclictest25664-21cut07:40:153
1038899118,2cyclictest24656-21awk07:40:003
1038899118,2cyclictest22834-21ssh11:36:063
1038899118,2cyclictest22834-21ssh11:36:063
1038899118,2cyclictest20618-21sh09:39:553
1038899118,2cyclictest14879-21seq12:30:023
1038899118,2cyclictest14594-21tail12:10:163
1038899118,2cyclictest13262-21tail08:20:253
1038899117,3cyclictest3766-21grep08:00:263
1038699118,2cyclictest7958-21cat08:10:212
1038699118,2cyclictest7370-21ssh09:31:022
1038699118,2cyclictest4676-21grep11:45:152
1038699118,2cyclictest4284-21ssh10:46:312
1038699118,2cyclictest27338-21ssh12:17:462
1038699118,2cyclictest25027-21ntpq08:45:212
1038699118,2cyclictest24386-21ssh09:22:032
1038699118,2cyclictest19861-21uname08:35:192
1038699118,2cyclictest1614-21chrt09:47:232
1038699118,2cyclictest16059-21ssh11:51:292
1038699118,2cyclictest14407-21rm09:15:442
1038699118,2cyclictest12689-21ssh09:35:032
1038699118,2cyclictest10013-21chrt10:11:282
1038699115,2cyclictest225-21systemd-journal11:40:172
1038399118,2cyclictest7545-21head10:10:161
1038399118,2cyclictest411-21ssh12:01:331
1038399118,2cyclictest29880-21grep09:45:151
1038399118,2cyclictest27383-21sh10:02:481
1038399118,2cyclictest21659-21expr08:40:151
1038399118,2cyclictest21237-21wc11:55:141
1038399118,2cyclictest19440-21sed10:55:241
1038399118,2cyclictest17048-21sed12:30:261
1038399118,2cyclictest1252-21grep09:05:141
1038399118,2cyclictest11187-21chrt07:10:151
1038399116,4cyclictest23743-21head07:35:201
1038399114,4cyclictest225-21systemd-journal12:35:191
1038199119,1cyclictest31693-21grep09:00:180
1038199119,1cyclictest10295-21ssh11:10:030
1038199118,2cyclictest7026-21tr12:25:140
1038199118,2cyclictest4527-21grep09:10:160
1038199118,2cyclictest28542-21ssh11:59:430
1038199118,2cyclictest24174-21awk07:35:240
1038199118,2cyclictest19001-21sh10:16:580
1038199118,2cyclictest17385-21chrt07:23:440
1038199118,2cyclictest17381-21ssh10:55:040
1038199118,2cyclictest16422-21ssh11:32:370
1038199118,2cyclictest14843-21chrt07:16:520
1038199118,2cyclictest14207-21ssh09:15:350
1038199117,3cyclictest6576-21cat08:10:000
1038199113,2cyclictest225-21systemd-journal07:44:550
1038199111,3cyclictest0-21swapper/011:20:250
1038199110,10cyclictest4201-21chrt11:05:360
1038899108,1cyclictest110352sleep308:16:443
1038899107,2cyclictest15875-21sed07:20:143
1038699108,1cyclictest30503-21mii-tool07:50:182
1038699107,2cyclictest9389-21cat08:15:142
1038699107,2cyclictest27516-21grep07:45:142
1038699107,2cyclictest25665-21tail07:40:152
1038699107,2cyclictest18670-21munin-run08:35:002
1038699100,9cyclictest3125-21taskset09:09:332
1038699100,9cyclictest0-21swapper/208:40:122
1038699100,3cyclictest3846-21runrttasks08:01:012
1038699100,2cyclictest0-21swapper/207:10:142
1038399108,1cyclictest8764-21seq08:13:091
1038399108,1cyclictest32473-21expr07:55:121
1038399108,1cyclictest29878-21grep07:50:141
1038399108,1cyclictest28436-21ssh11:40:131
1038399108,1cyclictest24859-21taskset07:40:001
1038399108,1cyclictest14109-21cpu08:25:141
1038399107,2cyclictest4906-21ssh10:28:121
1038399107,2cyclictest2392-21expr08:00:151
1038399107,2cyclictest16916-21expr08:30:131
1038399107,2cyclictest13482-21df07:15:161
1038199108,1cyclictest23897-21cat08:45:110
1038199108,1cyclictest2245-21grep08:00:130
1038199108,1cyclictest19595-21chrt07:26:380
1038199108,1cyclictest12604-21load08:20:190
1038199107,2cyclictest28024-21grep07:45:170
1038199107,2cyclictest20064-21ssh10:36:380
1038199104,2cyclictest225-21systemd-journal08:14:540
103889996,2cyclictest15613-21grep08:25:263
103889990,8cyclictest0-21swapper/307:50:123
103869997,1cyclictest5409-21cat08:05:192
103869990,4cyclictest32732-21grep09:00:272
103839998,1cyclictest6386-21taskset08:08:051
103839998,1cyclictest20498-21ls08:35:261
103839997,1cyclictest177242sleep107:25:061
103839996,2cyclictest9579-21expr08:15:161
103819997,1cyclictest91432sleep008:15:020
103819996,2cyclictest30110-21ls07:50:150
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional