You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-17 - 14:36

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot7s.osadl.org (updated Sun May 17, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
700921020,0sleep00-21swapper/010:46:360
1178329280,8sleep00-21swapper/007:07:020
1186329063,9sleep30-21swapper/307:08:053
1177928663,18sleep10-21swapper/107:06:591
1174828668,14sleep20-21swapper/207:06:362
281972720,1sleep013-21ksoftirqd/011:18:540
281972720,1sleep013-21ksoftirqd/011:18:540
152392620,1sleep10-21swapper/112:09:161
1209699290,28cyclictest0-21swapper/311:57:223
12087992928,1cyclictest0-21swapper/111:59:131
12086992827,1cyclictest0-21swapper/012:27:330
1208699270,4cyclictest0-21swapper/010:09:160
1209699241,11cyclictest0-21swapper/311:10:153
14642230,0sleep20-21swapper/212:20:152
1208799231,1cyclictest5162-21diskmemload09:44:311
1208799230,19cyclictest0-21swapper/109:10:471
1208699220,1cyclictest0-21swapper/011:07:320
1208799210,1cyclictest0-21swapper/112:00:301
1208799210,0cyclictest0-21swapper/111:31:071
12087992018,1cyclictest0-21swapper/111:15:171
12087992018,1cyclictest0-21swapper/111:15:171
1208799200,3cyclictest0-21swapper/111:25:141
1208699190,15cyclictest0-21swapper/010:56:590
297302180,0sleep10-21swapper/109:24:341
12096991816,2cyclictest0-21swapper/312:26:243
1208799182,15cyclictest12336-21apt-get07:10:151
12087991818,0cyclictest0-21swapper/110:53:551
1208799180,2cyclictest0-21swapper/111:06:131
12086991817,1cyclictest0-21swapper/011:04:070
12086991816,1cyclictest13-21ksoftirqd/010:20:580
12096991716,1cyclictest0-21swapper/310:10:153
1209699170,1cyclictest0-21swapper/310:18:503
1209199172,9cyclictest27007-21idleruntime-cro10:40:002
1208799170,17cyclictest0-21swapper/110:09:101
1208699175,7cyclictest13267-21awk08:20:000
1209699167,8cyclictest537-21snmpd09:29:553
12096991616,0cyclictest0-21swapper/311:06:503
1209699161,5cyclictest141rcu_preempt09:42:053
12096991613,1cyclictest0-21swapper/310:55:493
1209699160,1cyclictest0-21swapper/309:58:093
1209199163,5cyclictest0-21swapper/211:20:142
1209199161,9cyclictest26393-21apache_volume12:35:122
1209199160,14cyclictest0-21swapper/209:21:112
12087991614,1cyclictest0-21swapper/108:30:141
1208799160,2cyclictest0-21swapper/110:00:081
1208799160,2cyclictest0-21swapper/110:00:081
1208799160,16cyclictest0-21swapper/112:34:361
1208699166,3cyclictest141rcu_preempt10:36:300
1208699160,1cyclictest0-21swapper/009:33:030
1208699160,16cyclictest0-21swapper/012:01:560
1209199153,5cyclictest5162-21diskmemload10:55:102
1209199152,6cyclictest225-21systemd-journal11:45:002
12091991514,1cyclictest0-21swapper/208:00:162
1209199150,14cyclictest0-21swapper/212:03:352
1208799158,2cyclictest4666-21sed09:05:261
12087991512,2cyclictest0-21swapper/111:39:291
1208799151,14cyclictest0-21swapper/109:48:141
1208799150,15cyclictest0-21swapper/112:39:301
1208799150,15cyclictest0-21swapper/111:54:591
1208699150,2cyclictest0-21swapper/011:25:290
1208699150,1cyclictest0-21swapper/009:50:550
1209699141,9cyclictest0-21swapper/312:10:143
12096991413,1cyclictest0-21swapper/310:52:493
12096991412,1cyclictest0-21swapper/309:30:413
12096991410,3cyclictest9975-21wc09:50:253
12096991410,3cyclictest26467-21cat07:40:003
12096991410,3cyclictest16559-21grep11:30:303
1209699140,1cyclictest0-21swapper/310:40:153
1209699140,1cyclictest0-21swapper/310:32:583
1209699140,13cyclictest8350-21wc11:45:193
1209199149,3cyclictest18391-21ssh12:30:142
1209199146,6cyclictest31834-21apache210:26:272
1209199143,10cyclictest55750irq/128-enp2s0-09:12:372
12091991411,2cyclictest19918-21cron10:55:002
12091991410,2cyclictest16056-21fgrep11:30:242
1209199140,3cyclictest0-21swapper/212:29:442
1209199140,3cyclictest0-21swapper/211:19:362
1209199140,3cyclictest0-21swapper/211:19:362
1209199140,1cyclictest0-21swapper/212:16:542
1209199140,14cyclictest0-21swapper/209:48:362
1209199140,13cyclictest0-21swapper/211:29:052
1208799141,1cyclictest16041-21ssh11:10:571
1208799141,12cyclictest21168-21ssh09:57:131
1208799141,12cyclictest0-21swapper/110:55:371
12087991410,3cyclictest5567-21wc08:00:291
12087991410,3cyclictest31946-21ssh11:20:271
12087991410,2cyclictest513-21cron12:20:001
12087991410,2cyclictest31635-21tr07:50:141
12087991410,2cyclictest0-21swapper/112:12:521
1208699143,6cyclictest0-21swapper/007:10:150
1208699143,10cyclictest11984-21/usr/sbin/munin10:50:110
12086991412,1cyclictest0-21swapper/010:40:280
12086991410,3cyclictest23202-21tail09:20:160
1208699140,14cyclictest0-21swapper/011:55:200
1208699140,12cyclictest25484-21sort08:45:010
12096991310,2cyclictest21640-21head08:35:163
12096991310,2cyclictest18361-21wc07:20:213
12096991310,2cyclictest10445-21head08:10:283
1209699130,2cyclictest0-21swapper/312:17:233
1209699130,1cyclictest0-21swapper/309:19:183
1209199139,3cyclictest8020-21sed08:05:272
1209199139,3cyclictest29086-21cut11:00:152
1209199139,3cyclictest0-21swapper/211:10:222
1209199139,2cyclictest28986-21wc10:40:232
1209199139,2cyclictest14120-21grep08:20:162
1209199138,4cyclictest12051-21ssh11:09:312
1209199138,3cyclictest16403-21grep11:50:172
12091991310,2cyclictest32341-21grep08:55:272
12091991310,2cyclictest31622-21sed09:25:162
12091991310,2cyclictest28305-21grep10:20:262
12091991310,2cyclictest21650-21cut08:35:162
12091991310,2cyclictest20574-21cat07:25:202
12091991310,2cyclictest14899-21head09:15:182
1209199130,12cyclictest0-21swapper/211:38:332
1208799132,4cyclictest0-21swapper/111:00:141
12087991313,0cyclictest0-21swapper/108:50:121
1208799131,11cyclictest15149-21sh12:28:081
12087991310,2cyclictest8676-21wc11:45:211
12087991310,2cyclictest751-21grep11:40:261
12087991310,2cyclictest32091-21cut09:25:191
12087991310,2cyclictest27950-21wc07:40:241
12087991310,2cyclictest22514-21grep08:35:261
1208799130,8cyclictest0-21swapper/108:05:121
1208799130,1cyclictest0-21swapper/107:29:341
1208799130,12cyclictest1898-21grep12:20:161
1208699139,3cyclictest29627-21sh11:39:090
1208699139,3cyclictest27275-21grep07:40:190
1208699139,3cyclictest19516-21grep07:25:140
1208699138,3cyclictest27244-21sed12:35:170
1208699138,3cyclictest1475-21grep09:45:250
12086991312,1cyclictest0-21swapper/010:00:590
12086991312,1cyclictest0-21swapper/010:00:590
12086991312,1cyclictest0-21swapper/009:25:120
12086991311,1cyclictest0-21swapper/010:15:170
12086991310,2cyclictest8093-21sed11:45:170
12086991310,2cyclictest7508-21grep09:10:250
12086991310,2cyclictest31098-21cron07:50:000
1208699130,12cyclictest0-21swapper/011:33:400
1209699129,3cyclictest26412-21kworker/u8:112:22:343
1209699129,2cyclictest3875-21wc10:25:223
1209699129,2cyclictest32133-21tail11:20:273
1209699129,2cyclictest31716-21grep08:55:203
1209699129,2cyclictest29902-21ssh11:00:203
1209699129,2cyclictest28453-21tr08:50:143
1209699129,2cyclictest2722-21awk07:55:203
1209699129,2cyclictest243282sleep311:15:523
1209699129,2cyclictest243282sleep311:15:523
1209699129,2cyclictest23971-21turbostat07:35:003
1209699129,2cyclictest23119-21sendmail-msp09:40:003
1209699129,2cyclictest21595-21dump-pmu-power07:30:003
1209699129,2cyclictest11898-21grep08:15:163
1209699128,3cyclictest28291-21sed07:40:283
1209699128,3cyclictest21051-21sed10:35:253
1209699126,3cyclictest225-21systemd-journal12:10:003
1209699120,11cyclictest28005-21basename08:50:003
1209199129,2cyclictest411-21wc07:50:242
1209199129,2cyclictest3246-21grep07:55:262
1209199129,2cyclictest30208-21cut07:45:222
1209199129,2cyclictest24855-21cut07:35:152
1209199129,2cyclictest2134-21head10:05:162
1209199129,2cyclictest18066-21ssh10:34:322
1209199129,2cyclictest17628-21sort09:55:172
1209199128,3cyclictest9982-21tr10:10:162
1209199128,3cyclictest17732-21sort07:20:162
1209199127,4cyclictest8905-21ssh10:48:062
1209199120,12cyclictest0-21swapper/212:10:502
1209199120,12cyclictest0-21swapper/212:05:212
1209199120,11cyclictest270742sleep209:41:342
1208799129,2cyclictest8426-21grep09:50:141
1208799129,2cyclictest17846-21tail07:20:161
1208799129,2cyclictest15943-21ssh09:35:171
1208799129,2cyclictest15929-21sed07:15:221
1208799129,2cyclictest11220-21cut08:15:131
1208799128,3cyclictest7064-21tr09:30:141
1208799128,3cyclictest18782-21ssh10:35:001
1208799128,3cyclictest1758-21ssh10:24:521
1208799120,12cyclictest0-21swapper/110:29:481
1208799120,12cyclictest0-21swapper/109:17:261
1208799120,11cyclictest3519-21ssh10:45:131
1208699129,2cyclictest32642-21ls07:50:210
1208699129,2cyclictest28904-21cut08:50:160
1208699129,2cyclictest25815-21grep12:15:160
1208699129,2cyclictest22577-21grep07:30:150
1208699129,2cyclictest18674-21cat07:20:250
1208699129,2cyclictest17063-21wc12:10:150
1208699129,2cyclictest10168-21grep12:05:250
1208699128,3cyclictest3301-21tr09:05:140
1208699128,3cyclictest2339-21grep12:20:200
1208699124,4cyclictest225-21systemd-journal11:25:000
1208699124,4cyclictest225-21systemd-journal08:00:000
1208699123,7cyclictest5162-21diskmemload10:25:160
1208699122,9cyclictest31502-21perf09:45:000
1209699118,2cyclictest9993-21ssh09:12:153
1209699118,2cyclictest6484-21tr11:25:163
1209699118,2cyclictest5352-21cat10:45:263
1209699118,2cyclictest32036-21chrt09:45:123
1209699118,2cyclictest3145-21tr09:05:143
1209699118,2cyclictest30353-21ssh12:36:573
1209699118,2cyclictest2788-21ssh12:01:053
1209699118,2cyclictest27501-21ssh10:00:373
1209699118,2cyclictest27501-21ssh10:00:373
1209699118,2cyclictest25775-21ssh09:21:293
1209699118,2cyclictest23485-21ls08:40:133
1209699118,2cyclictest22154-21ssh12:32:043
1209699118,2cyclictest20792-21sshd11:52:573
1209699118,2cyclictest17738-21cat08:25:263
1209699118,2cyclictest15405-21tune2fs07:15:183
1209699118,2cyclictest13659-21grep08:20:143
1209699118,2cyclictest13462-21tr07:10:213
1209699118,2cyclictest1153-21chrt10:05:083
1209699117,3cyclictest32181-21grep07:50:163
1209699117,3cyclictest31337-21cut11:40:143
1209699117,2cyclictest22653-21rm11:35:043
1209199118,2cyclictest9761-21ssh09:31:112
1209199118,2cyclictest27469-21ssh11:56:372
1209199118,2cyclictest27151-21cat10:00:282
1209199118,2cyclictest27151-21cat10:00:282
1209199118,2cyclictest26633-21dpkg07:40:152
1209199118,2cyclictest26083-21tr08:45:152
1209199118,2cyclictest23542-21grep07:30:252
1209199118,2cyclictest21593-21ssh10:16:402
1209199118,2cyclictest19409-21ssh09:37:092
1209199118,2cyclictest16440-21tr08:25:152
1209199118,2cyclictest16055-21basename07:15:232
1209199118,2cyclictest12457-21aten2_r7power_v07:10:152
1209199118,2cyclictest10451-21taskset11:46:232
1209199117,3cyclictest2276-21grep09:00:282
1209199111,6cyclictest225-21systemd-journal08:20:002
1208799118,2cyclictest2754-21taskset09:04:541
1208799118,2cyclictest20186-21ssh10:35:181
1208799118,2cyclictest18835-21ssh10:15:151
1208799118,2cyclictest1642-21cut07:55:131
1208799117,3cyclictest30265-21sh10:41:011
1208799117,3cyclictest11135-21wc10:10:241
1208799113,4cyclictest225-21systemd-journal08:45:011
1208699118,2cyclictest32014-21grep11:40:170
1208699118,2cyclictest31174-21sed08:55:160
1208699118,2cyclictest20606-21sh09:19:000
1208699118,2cyclictest18793-21grep12:30:160
1208699118,2cyclictest17317-21ssh09:35:310
1208699118,2cyclictest16881-21grep08:25:170
1208699118,2cyclictest16840-21munin-run07:20:000
1208699118,2cyclictest13198-21ssh11:10:030
1208699118,2cyclictest10765-21sort10:10:200
1208699117,3cyclictest7985-21grep08:05:270
1208699110,1cyclictest141rcu_preempt09:57:230
1208699110,10cyclictest19021-21sh11:51:340
1209699108,1cyclictest4348-21taskset08:00:183
1209699108,1cyclictest26805-21expr10:20:153
1209699107,2cyclictest902-21cut09:00:163
1209699107,2cyclictest7123-21grep08:05:173
1209699107,2cyclictest20138-21sed08:30:263
1209699100,9cyclictest289662sleep307:45:113
1209199108,1cyclictest9720-21cat08:10:202
1209199108,1cyclictest8838-21sed09:50:152
1209199108,1cyclictest4819-21chrt09:05:562
1209199108,1cyclictest0-21swapper/208:50:152
1209199107,2cyclictest23549-21expr08:40:132
1208799108,1cyclictest30845-21taskset07:46:511
1208799108,1cyclictest27364-21grep08:45:261
1208799108,1cyclictest22875-21memory07:30:181
1208799107,2cyclictest32097-21awk08:55:251
1208799107,2cyclictest14822-21cut08:20:221
1208799101,7cyclictest9007-21tr08:10:161
1208799101,5cyclictest225-21systemd-journal07:35:001
1208799100,9cyclictest180382sleep108:27:161
1208699108,1cyclictest858-21grep09:00:160
1208699108,1cyclictest25440-21cut07:35:200
1208699108,1cyclictest20130-21unixbench-2d08:30:260
1208699108,1cyclictest13527-21apt-get08:20:130
1208699108,1cyclictest10798-21ssh10:30:120
1208699107,2cyclictest9343-21sort08:10:180
1208699107,2cyclictest26280-21expr08:45:150
1208699105,3cyclictest4263-21cut08:00:180
120919996,2cyclictest18888-21expr08:30:142
120869997,1cyclictest227442sleep008:36:450
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional