You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-24 - 22:05

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot7s.osadl.org (updated Thu Sep 24, 2026 12:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
677229567,9sleep00-21swapper/007:05:280
688529467,9sleep30-21swapper/307:06:493
499729077,9sleep20-21swapper/207:05:032
705328164,13sleep10-21swapper/107:09:001
171702550,0sleep00-21swapper/010:36:540
722299271,2cyclictest141rcu_preempt10:38:031
722799251,1cyclictest0-21swapper/310:37:213
7222992523,2cyclictest25-21ksoftirqd/111:23:411
722299250,1cyclictest0-21swapper/110:57:491
7224992423,1cyclictest0-21swapper/210:15:052
722299243,5cyclictest141rcu_preempt11:07:181
722299242,8cyclictest25-21ksoftirqd/112:13:381
7222992423,1cyclictest0-21swapper/110:11:391
7222992419,4cyclictest29739-21runrttasks11:43:061
7227992317,3cyclictest39-21ksoftirqd/311:26:083
7222992320,2cyclictest0-21swapper/111:50:421
722299231,1cyclictest241rcuc/107:30:191
722299230,3cyclictest0-21swapper/112:38:131
722299230,1cyclictest0-21swapper/112:08:551
722299230,19cyclictest0-21swapper/109:37:481
722299230,19cyclictest0-21swapper/109:37:481
722499220,0cyclictest0-21swapper/211:37:352
722299220,22cyclictest0-21swapper/110:29:311
722299220,21cyclictest0-21swapper/109:42:431
7227992116,3cyclictest39-21ksoftirqd/307:25:013
722799210,3cyclictest141rcu_preempt12:08:113
7222992120,1cyclictest25-21ksoftirqd/109:56:231
7222992120,1cyclictest0-21swapper/109:24:081
722299211,14cyclictest16011-21kworker/1:110:40:161
722799208,9cyclictest39-21ksoftirqd/311:08:483
7227992018,1cyclictest0-21swapper/307:15:133
7227992016,3cyclictest29367-21ssh10:44:583
7227992012,4cyclictest39-21ksoftirqd/312:35:093
722799201,1cyclictest141rcu_preempt10:30:193
722799201,19cyclictest381rcuc/312:32:303
7222992019,1cyclictest0-21swapper/111:46:201
722299201,18cyclictest2530-21aten2_r7power_v12:25:131
722299200,1cyclictest0-21swapper/112:32:301
722299200,17cyclictest0-21swapper/110:23:121
722799192,3cyclictest141rcu_preempt11:34:483
722799192,2cyclictest141rcu_preempt09:25:463
722499194,14cyclictest0-21swapper/210:27:512
722499191,3cyclictest0-21swapper/211:56:542
722499190,0cyclictest0-21swapper/209:45:092
7222991917,1cyclictest38150irq/131-snd_hda11:58:531
721799195,7cyclictest24790-21sendmail12:00:000
722799183,2cyclictest141rcu_preempt12:28:263
722799181,4cyclictest39-21ksoftirqd/309:33:103
7227991813,2cyclictest381rcuc/310:59:473
722799180,0cyclictest141rcu_preempt12:10:343
7224991818,0cyclictest0-21swapper/209:43:572
722299183,15cyclictest0-21swapper/110:51:361
7222991816,2cyclictest25-21ksoftirqd/111:35:121
7222991816,2cyclictest25-21ksoftirqd/109:15:321
7222991816,1cyclictest0-21swapper/110:15:051
7222991815,2cyclictest25-21ksoftirqd/111:19:551
7222991810,8cyclictest25-21ksoftirqd/108:05:101
722799175,2cyclictest39-21ksoftirqd/310:00:203
7227991716,1cyclictest0-21swapper/310:11:383
722799171,1cyclictest537-21snmpd12:17:463
722799170,1cyclictest0-21swapper/311:22:443
722499170,1cyclictest0-21swapper/211:11:522
722499170,0cyclictest0-21swapper/212:25:372
7222991716,1cyclictest0-21swapper/111:11:091
7222991715,2cyclictest32698-21diskmemload09:54:071
7222991715,1cyclictest0-21swapper/111:29:331
722299170,1cyclictest0-21swapper/111:33:201
722299170,10cyclictest0-21swapper/112:17:311
721799175,6cyclictest30333-21munin-run11:05:000
721799175,6cyclictest30333-21munin-run11:05:000
721799174,8cyclictest18947-21irqstats12:15:170
721799170,3cyclictest0-21swapper/011:34:320
722799161,5cyclictest32698-21diskmemload11:59:253
7227991614,1cyclictest0-21swapper/311:50:423
722799161,2cyclictest141rcu_preempt10:27:393
7227991612,2cyclictest39-21ksoftirqd/310:51:063
722799160,4cyclictest0-21swapper/309:48:403
722799160,16cyclictest0-21swapper/309:41:453
7224991616,0cyclictest0-21swapper/210:37:222
722499161,1cyclictest0-21swapper/207:15:202
722499160,1cyclictest0-21swapper/212:16:532
722499160,15cyclictest0-21swapper/210:33:572
722299162,13cyclictest0-21swapper/110:45:251
722299161,10cyclictest0-21swapper/109:48:021
722299160,2cyclictest0-21swapper/109:28:001
721799163,7cyclictest29495-21sed10:44:590
721799161,11cyclictest4168-21runrttasks10:29:490
92202150,0chrt0-21swapper/210:13:042
722799156,3cyclictest39-21ksoftirqd/311:49:593
7227991514,1cyclictest381rcuc/309:50:203
7227991512,1cyclictest39-21ksoftirqd/309:59:223
722799151,12cyclictest23505-21ssh09:23:343
722799150,1cyclictest0-21swapper/308:59:593
722799150,11cyclictest0-21swapper/309:15:063
722499150,15cyclictest0-21swapper/212:13:182
722299154,10cyclictest25-21ksoftirqd/110:30:161
722299153,9cyclictest25-21ksoftirqd/109:31:491
722299151,12cyclictest25-21ksoftirqd/108:55:141
722299150,8cyclictest0-21swapper/107:15:121
721799152,6cyclictest734-21perf12:05:000
722799148,4cyclictest17251-21apt-get07:30:133
7227991413,1cyclictest39-21ksoftirqd/310:48:283
722499145,7cyclictest2847-21grep09:50:132
7224991411,2cyclictest11402-21tr09:55:142
722499140,1cyclictest0-21swapper/210:50:262
722299149,4cyclictest26498-21sh07:50:001
722299147,7cyclictest32698-21diskmemload09:12:521
722299144,9cyclictest0-21swapper/108:25:131
7222991412,1cyclictest0-21swapper/112:25:001
7222991410,2cyclictest25-21ksoftirqd/108:34:591
721799146,7cyclictest9074-21munin-run12:10:000
721799143,7cyclictest225-21systemd-journal10:15:000
7217991411,2cyclictest11433-21wc07:15:250
7217991410,3cyclictest10116-21grep09:35:130
7217991410,3cyclictest10116-21grep09:35:120
7217991410,2cyclictest5981-21turbostat08:15:010
7217991410,2cyclictest2178-21grep11:45:260
721799140,1cyclictest0-21swapper/009:48:080
722799131,2cyclictest0-21swapper/312:03:293
7227991312,1cyclictest39-21ksoftirqd/308:20:153
7227991312,1cyclictest381rcuc/311:13:363
7227991311,2cyclictest39-21ksoftirqd/311:18:313
722799131,10cyclictest6228-21ssh09:13:073
7227991310,2cyclictest26219-21grep11:40:243
7227991310,2cyclictest19468-21sed08:40:153
7227991310,2cyclictest16165-21wget11:35:153
7227991310,2cyclictest13316-21grep10:15:183
722799130,3cyclictest141rcu_preempt09:38:093
722799130,3cyclictest141rcu_preempt09:38:083
722499139,3cyclictest20297-21tail10:00:162
722499138,5cyclictest291irq_work/211:47:452
722499138,4cyclictest21906-21taskset11:19:132
722499131,8cyclictest0-21swapper/210:55:152
722499131,1cyclictest0-21swapper/210:41:122
7224991310,2cyclictest8208-21sed07:10:182
7224991310,2cyclictest29819-21grep09:00:282
7224991310,2cyclictest27928-21ssh10:05:112
7224991310,2cyclictest26180-21grep09:25:142
7224991310,2cyclictest1932-21basename09:30:142
7224991310,2cyclictest15498-21grep07:25:162
7224991310,2cyclictest14473-21grep08:30:172
7224991310,2cyclictest12392-21grep07:20:142
722299133,3cyclictest25-21ksoftirqd/110:05:241
7222991312,1cyclictest0-21swapper/112:01:081
7222991310,2cyclictest27275-21cron10:05:001
7222991310,2cyclictest20172-21wc08:40:231
722299130,2cyclictest225-21systemd-journal07:29:551
721799133,2cyclictest0-21swapper/010:30:180
7217991311,2cyclictest13-21ksoftirqd/009:34:060
7217991310,2cyclictest8812-21ssh11:50:150
7217991310,2cyclictest7566-21grep11:10:190
7217991310,2cyclictest28888-21cut09:00:180
7217991310,2cyclictest21355-21grep08:45:130
7217991310,2cyclictest20243-21tune2fs07:35:170
7217991310,2cyclictest20243-21tune2fs07:35:170
7217991310,2cyclictest12979-21grep09:55:260
721799130,12cyclictest18855-21ssh10:18:520
722799129,2cyclictest31030-21taskset07:57:193
722799129,2cyclictest22752-21tr11:00:173
722799129,2cyclictest22752-21tr11:00:163
722799129,2cyclictest21349-21grep08:45:143
722799129,2cyclictest15395-21sed08:30:263
722799128,2cyclictest12587-21wc08:25:253
722799125,3cyclictest39-21ksoftirqd/312:21:213
722799125,2cyclictest141rcu_preempt08:35:003
722799122,2cyclictest537-21snmpd07:42:133
7227991210,2cyclictest39-21ksoftirqd/310:08:063
722799120,11cyclictest22336-21rm10:20:223
722499129,2cyclictest32699-21dump-pmu-power09:10:002
722499129,2cyclictest29929-21tail07:55:172
722499129,2cyclictest22796-21grep07:40:172
722499129,2cyclictest2023-21grep12:05:162
722499129,2cyclictest17948-21cron09:40:002
722499129,2cyclictest17948-21cron09:40:002
722499128,3cyclictest19778-21grep08:40:172
722499128,3cyclictest19544-21cat07:35:012
722499128,1cyclictest14701-21chrt11:53:382
722499123,5cyclictest225-21systemd-journal12:35:002
722499120,12cyclictest0-21swapper/209:16:352
722499120,0cyclictest0-21swapper/211:04:242
722499120,0cyclictest0-21swapper/211:04:232
722299125,4cyclictest25-21ksoftirqd/107:40:471
722299124,4cyclictest225-21systemd-journal11:03:041
722299124,4cyclictest225-21systemd-journal11:03:041
722299120,1cyclictest0-21swapper/109:00:221
722299120,11cyclictest21136-21tr08:45:121
721799129,2cyclictest9842-21sort09:15:160
721799129,2cyclictest4174-21wc09:50:210
721799129,2cyclictest31455-21cat08:00:010
721799129,2cyclictest2690-21grep12:25:130
721799129,2cyclictest23472-21cut08:50:120
721799129,2cyclictest20731-21ssh12:35:210
721799128,3cyclictest24221-21sed11:20:150
721799127,4cyclictest10416-21apache212:22:000
721799127,2cyclictest0-21swapper/009:14:160
721799126,4cyclictest27913-21cron09:00:000
721799126,4cyclictest16660-21ssh11:15:210
721799125,3cyclictest225-21systemd-journal09:45:000
721799123,2cyclictest22844-21taskset10:01:310
721799121,6cyclictest20824-21idleruntime-cro08:45:000
721799120,11cyclictest22693-21ssh10:20:260
722799118,2cyclictest20588-21wc07:35:193
722799118,2cyclictest20588-21wc07:35:183
722799118,1cyclictest28682-21taskset07:52:413
722799116,2cyclictest225-21systemd-journal08:15:013
722799110,4cyclictest225-21systemd-journal07:47:413
722499119,1cyclictest21091-21cut10:20:142
722499118,2cyclictest8981-21cat08:20:152
722499118,2cyclictest8037-21ssh09:14:332
722499118,2cyclictest6914-21grep08:15:152
722499118,2cyclictest5061-21pmu-power08:10:212
722499118,2cyclictest30919-21grep11:05:152
722499118,2cyclictest29273-21ssh11:42:432
722499118,2cyclictest28544-21ssh11:22:522
722499118,2cyclictest27240-21sort12:20:142
722499118,2cyclictest27016-21tr07:50:132
722499118,2cyclictest25694-21wc08:55:002
722499118,2cyclictest25382-21sh12:00:142
722499118,2cyclictest23238-21ssh12:36:532
722499118,2cyclictest21937-21cat08:45:172
722499118,2cyclictest20635-21ssh09:21:182
722499118,2cyclictest16535-21tr08:35:152
722499118,2cyclictest1274-21grep11:25:262
722499110,2cyclictest20623-21kworker/u8:010:48:372
722499110,10cyclictest26218-21taskset07:46:282
722499110,0cyclictest0-21swapper/211:30:412
722299119,1cyclictest32157-21cut08:00:171
722299118,2cyclictest9901-21ps08:20:221
722299118,2cyclictest5889-21taskset08:14:041
722299114,4cyclictest25-21ksoftirqd/107:38:401
722299114,4cyclictest25-21ksoftirqd/107:38:401
722299113,7cyclictest25-21ksoftirqd/107:10:141
722299112,4cyclictest225-21systemd-journal08:17:461
722299112,1cyclictest141rcu_preempt07:50:001
721799118,2cyclictest9608-21ssh12:10:150
721799118,2cyclictest6125-21ssh10:50:200
721799118,2cyclictest32674-21idleruntime11:25:170
721799118,2cyclictest2688-21rm10:09:280
721799118,2cyclictest24505-21cut11:40:130
721799118,2cyclictest22298-21tr07:40:150
721799118,2cyclictest20615-21ssh09:21:170
721799118,2cyclictest18394-21cron08:40:000
721799118,2cyclictest17377-21ssh10:56:210
721799118,2cyclictest16744-21grep11:35:170
721799118,2cyclictest11187-21grep12:30:170
721799117,3cyclictest13946-21tr08:30:150
721799114,3cyclictest225-21systemd-journal08:10:000
721799112,4cyclictest0-21swapper/007:25:130
7217991111,0cyclictest0-21swapper/009:25:130
721799110,4cyclictest225-21systemd-journal07:34:590
721799110,10cyclictest30918-21wget11:05:150
721799110,10cyclictest0-21swapper/008:20:120
722799108,2cyclictest2614-21ntpq08:05:183
722799108,1cyclictest8378-21systemctl08:19:593
722799108,1cyclictest31265-21cut09:05:183
722799108,1cyclictest25043-21unixbench-2d08:50:243
722799107,2cyclictest1215-21cron08:05:013
722799105,4cyclictest39-21ksoftirqd/309:00:163
722499108,1cyclictest31901-21expr08:00:152
722499108,1cyclictest26958-21ntp_offset08:55:212
722499108,1cyclictest19821-21wc07:35:142
722499108,1cyclictest19821-21wc07:35:132
722499102,2cyclictest225-21systemd-journal08:05:142
722299108,1cyclictest31111-21sed09:05:171
722299108,1cyclictest23678-21tr08:50:131
722299101,8cyclictest38150irq/131-snd_hda07:20:141
722299100,6cyclictest25-21ksoftirqd/107:55:141
721799108,1cyclictest8824-21tr07:10:240
721799108,1cyclictest29906-21ssh10:45:110
721799108,1cyclictest27689-21uname07:50:180
721799108,1cyclictest24862-21expr07:45:140
721799108,1cyclictest13476-21awk07:20:210
721799107,2cyclictest32402-21taskset09:06:220
721799107,2cyclictest31747-21cat08:00:150
721799107,2cyclictest11157-21basename08:25:140
721799100,9cyclictest6382-21grep08:15:130
72279998,1cyclictest14813-21gpgconf07:25:013
72279997,2cyclictest7531-21aten2_r7power_p07:10:153
72249996,2cyclictest11751-21cut08:25:162
72229995,3cyclictest10371-21apache208:35:011
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional