You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-14 - 19:18

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Mon Sep 14, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2755829670,8sleep00-21swapper/007:08:080
2747128568,13sleep10-21swapper/107:07:011
2736328470,9sleep20-21swapper/207:05:372
2737628269,9sleep30-21swapper/307:05:463
83622380,1sleep30-21swapper/311:36:583
2779799260,26cyclictest0-21swapper/310:56:173
2779499261,24cyclictest0-21swapper/211:51:472
2779299262,23cyclictest0-21swapper/112:05:121
27787992620,2cyclictest225-21systemd-journal10:03:370
2778799250,1cyclictest0-21swapper/010:40:260
2779299232,18cyclictest3042-21aten2_r7power_p07:25:151
2779299230,0cyclictest0-21swapper/111:14:451
2779799220,22cyclictest0-21swapper/309:51:053
2778799220,2cyclictest0-21swapper/011:03:560
2779299212,3cyclictest24587-21ssh11:46:341
27792992115,3cyclictest141rcu_preempt12:37:371
2778799210,3cyclictest141rcu_preempt09:37:260
2779299203,16cyclictest12376-21diskstats11:20:151
8992180,0sleep30-21swapper/311:51:543
27794991817,1cyclictest0-21swapper/211:13:132
2779299180,15cyclictest0-21swapper/110:01:231
2779299171,2cyclictest0-21swapper/111:15:401
2778799174,6cyclictest8860-21idleruntime-cro08:45:000
2779799162,7cyclictest9624-21sshd07:37:383
2779499160,16cyclictest0-21swapper/212:30:422
2779499160,15cyclictest0-21swapper/207:10:142
2779299164,7cyclictest5328-21/usr/sbin/munin11:55:191
2779299164,7cyclictest5328-21/usr/sbin/munin11:55:191
27792991615,1cyclictest0-21swapper/109:24:361
27792991611,2cyclictest225-21systemd-journal12:18:271
2779299160,13cyclictest0-21swapper/109:35:431
27797991513,1cyclictest0-21swapper/310:25:143
2779799150,15cyclictest0-21swapper/311:06:423
2779499152,11cyclictest0-21swapper/211:15:402
2779499150,14cyclictest0-21swapper/209:40:302
2779299155,8cyclictest537-21snmpd09:58:131
2779299150,1cyclictest0-21swapper/110:55:151
2778799157,7cyclictest13203-21munin-run12:00:010
2778799157,7cyclictest13203-21munin-run12:00:000
2778799150,8cyclictest31202-21perf09:55:000
2778799150,1cyclictest0-21swapper/009:11:400
2778799150,13cyclictest0-21swapper/011:19:260
2779799148,4cyclictest39-21ksoftirqd/308:20:003
2779799142,1cyclictest141rcu_preempt09:23:183
27797991411,2cyclictest1712-21turbostat08:30:003
27794991414,0cyclictest0-21swapper/208:00:592
27794991410,2cyclictest7541-21wc12:15:202
2779499140,13cyclictest13134-21unixbench_multi08:50:252
2779499140,13cyclictest0-21swapper/210:29:432
2779499140,13cyclictest0-21swapper/210:17:082
27792991410,2cyclictest20797-21grep11:25:151
27792991410,2cyclictest17461-21grep10:05:231
2779299140,13cyclictest0-21swapper/111:43:361
2779299140,12cyclictest0-21swapper/110:35:581
2779799139,3cyclictest23837-21cron10:10:003
2779799139,3cyclictest14580-21cron07:49:593
2779799139,3cyclictest10834-21tail07:40:173
2779799134,8cyclictest31394-21ssh12:30:123
27797991312,1cyclictest0-21swapper/312:18:573
27797991310,2cyclictest39-21ksoftirqd/312:20:213
27797991310,2cyclictest21793-21idleruntime-cro08:05:003
2779799130,8cyclictest0-21swapper/308:10:123
2779499139,3cyclictest4103-21cat08:35:012
2779499139,3cyclictest30039-21cut11:30:222
2779499139,3cyclictest2498-21tr10:55:132
2779499139,3cyclictest15697-21sed07:50:162
2779499139,3cyclictest14985-21cut12:20:142
2779499139,2cyclictest6927-21tr08:40:122
2779499136,3cyclictest5461-21irqstats11:35:202
2779499134,2cyclictest22105-21apt-get12:05:122
27794991312,1cyclictest0-21swapper/210:38:422
27794991310,2cyclictest20782-21cron09:10:002
27794991310,2cyclictest18940-21grep07:55:272
2779299138,4cyclictest15853-21ssh12:00:301
2779299138,4cyclictest1344-21ssh12:30:431
27792991310,3cyclictest0-21swapper/109:48:441
27792991310,2cyclictest30322-21cat07:15:001
27792991310,2cyclictest23201-21grep09:10:271
27792991310,2cyclictest16446-21cut09:00:141
2779299130,1cyclictest0-21swapper/110:10:301
2778799139,3cyclictest14317-21chrt09:44:550
2778799139,3cyclictest10373-21grep07:40:150
2778799139,2cyclictest14155-21ls11:40:230
2778799135,7cyclictest0-21swapper/009:15:120
2778799132,6cyclictest27011-21idleruntime-cro11:10:010
27787991311,2cyclictest13-21ksoftirqd/010:13:440
27787991310,2cyclictest20206-21basename11:25:130
27787991310,2cyclictest13621-21wc08:55:000
2778799130,12cyclictest0-21swapper/011:50:120
2779799129,2cyclictest9790-21ssh11:57:203
2779799129,2cyclictest9790-21ssh11:57:203
2779799129,2cyclictest3654-21grep08:30:273
2779799129,2cyclictest3641-21cut07:25:173
2779799129,2cyclictest30347-21ssh11:30:253
2779799129,2cyclictest30192-21grep08:20:163
2779799129,2cyclictest22203-21cat11:45:173
2779799129,2cyclictest21562-21wc11:25:213
2779799129,2cyclictest1936-21cat10:35:173
2779799129,2cyclictest15809-21sed09:45:163
2779799129,2cyclictest13953-21ssh09:25:113
2779799129,2cyclictest12025-21tr11:20:143
2779799129,2cyclictest11230-21awk11:20:013
2779799124,4cyclictest225-21systemd-journal10:50:013
27797991211,1cyclictest0-21swapper/310:10:493
27797991210,2cyclictest0-21swapper/311:03:533
2779799120,11cyclictest31523-21head09:35:173
2779499129,2cyclictest808-21cat09:55:232
2779499129,2cyclictest32558-21grep08:25:152
2779499129,2cyclictest25824-21cut10:30:132
2779499129,2cyclictest13990-21grep08:55:132
2779499129,2cyclictest1323-21head07:20:162
2779499128,3cyclictest25277-21grep09:50:252
2779499128,3cyclictest14544-21head12:00:172
2779499128,1cyclictest7896-21tr10:00:132
2779499127,4cyclictest25844-21find10:10:202
2779499126,5cyclictest55850irq/129-enp2s0-12:28:572
2779499125,2cyclictest225-21systemd-journal10:44:562
2779499124,3cyclictest225-21systemd-journal10:23:182
2779499120,10cyclictest13211-21sendmail12:00:002
2779499120,10cyclictest13211-21sendmail12:00:002
2779499120,0cyclictest0-21swapper/209:22:082
2779299129,2cyclictest38150irq/131-snd_hda08:50:191
2779299129,2cyclictest32529-21cat07:19:591
2779299129,2cyclictest20347-21sort11:05:201
2779299129,2cyclictest20153-21grep10:45:251
2779299129,2cyclictest16564-21grep12:20:251
2779299129,2cyclictest16540-21sed07:50:261
2779299129,2cyclictest13246-21head07:45:171
2779299129,2cyclictest12312-21wc11:00:221
2779299129,2cyclictest11078-21ntpq10:40:201
2779299127,2cyclictest25343-21ssh09:31:131
2779299122,4cyclictest452-21systemd-logind11:37:031
2779299120,11cyclictest6585-21chrt09:40:111
2779299120,11cyclictest27294-21grep10:50:151
2779299120,11cyclictest23416-21chrt10:28:521
2778799129,2cyclictest9841-21run-parts07:39:590
2778799129,2cyclictest5735-21tr09:20:150
2778799129,2cyclictest31985-21tr09:55:160
2778799129,2cyclictest29731-21sed07:10:260
2778799129,2cyclictest27900-21tr10:50:200
2778799129,2cyclictest25821-21grep10:30:140
2778799129,2cyclictest21805-21perf08:05:000
2778799129,2cyclictest21437-21tr11:45:130
2778799129,2cyclictest1820-21grep10:35:160
2778799129,2cyclictest17093-21tr10:25:130
2778799129,2cyclictest15329-21sort12:20:150
2778799129,2cyclictest14582-21cron07:49:590
2778799128,3cyclictest14459-21grep12:00:170
2778799120,11cyclictest0-21swapper/011:37:370
2779799118,2cyclictest8636-21cut10:00:173
2779799118,2cyclictest814-21tr07:20:143
2779799118,2cyclictest6375-21sh09:40:043
2779799118,2cyclictest4784-21expr08:35:153
2779799118,2cyclictest30863-21grep09:15:183
2779799118,2cyclictest28010-21df11:10:133
2779799118,2cyclictest27164-21ssh09:32:393
2779799118,2cyclictest26206-21ssh10:30:143
2779799118,2cyclictest23781-21wc12:05:243
2779799118,2cyclictest22817-21expr08:05:183
2779799118,2cyclictest15547-21cat12:40:013
2779799118,2cyclictest1550-21ssh12:11:403
2779799118,2cyclictest1499-21ssh09:55:363
2779799118,2cyclictest14095-21tr08:55:133
2779799118,2cyclictest13735-21ssh10:41:573
2779799118,2cyclictest12638-21ssh11:40:083
2779799118,2cyclictest10793-21ssh10:20:333
2779799117,3cyclictest2010-21wc10:15:253
2779799110,10cyclictest72822sleep307:33:033
2779799110,10cyclictest22984-21rm10:47:373
2779799110,10cyclictest16829-21ssh12:01:153
2779499118,2cyclictest8287-21uniq12:35:172
2779499118,2cyclictest5097-21head08:35:172
2779499118,2cyclictest4325-21sh09:19:262
2779499118,2cyclictest32623-21cat07:19:592
2779499118,2cyclictest31541-21basename08:25:012
2779499118,2cyclictest27409-21sort08:15:172
2779499118,2cyclictest27305-21grep10:50:162
2779499118,2cyclictest26956-21ssh11:48:262
2779499118,2cyclictest24945-21ssh09:30:552
2779499118,2cyclictest24697-21tr08:10:142
2779499118,2cyclictest23983-21ssh09:10:592
2779499118,2cyclictest23093-21ssh11:26:112
2779499118,2cyclictest20486-21ssh10:45:392
2779499118,2cyclictest19264-21tr11:05:132
2779499118,2cyclictest18023-21sed10:05:282
2779499118,2cyclictest1617-21ssh09:36:292
2779499118,2cyclictest13579-21head11:40:182
2779499118,2cyclictest11339-21head07:40:222
2779499118,2cyclictest11016-21ssh11:00:142
2779499117,3cyclictest32117-21wc12:10:242
2779499110,3cyclictest0-21swapper/209:25:342
2779499110,10cyclictest11539-21sh11:20:042
2779299118,2cyclictest6182-21taskset08:36:431
2779299118,2cyclictest4090-21ssh10:16:521
2779299118,2cyclictest27052-21expr08:15:151
2779299118,2cyclictest26793-21ssh10:30:191
2779299118,2cyclictest2365-21ssh11:53:021
2779299118,2cyclictest2344-21ssh11:33:471
2779299118,2cyclictest22831-21ssh12:25:061
2779299118,2cyclictest21728-21taskset08:04:241
2779299118,2cyclictest2171-21cat08:30:151
2779299118,2cyclictest20788-21sh09:10:001
2779299118,2cyclictest17200-21ssh09:26:211
2779299118,2cyclictest1203-21ssh09:16:581
2779299117,3cyclictest9396-21grep07:35:281
2779299117,3cyclictest10384-21wc10:20:241
2779299117,2cyclictest5579-21tr07:30:131
2779299115,5cyclictest23743-21sed08:05:261
2779299110,3cyclictest0-21swapper/112:12:501
2779299110,10cyclictest23280-21basename09:50:101
2779299110,10cyclictest0-21swapper/108:10:131
2778799119,1cyclictest22773-21head08:05:170
2778799118,2cyclictest8545-21ssh10:20:090
2778799118,2cyclictest7275-21taskset07:32:570
2778799118,2cyclictest6886-21sed12:15:150
2778799118,2cyclictest4260-21ssh10:16:590
2778799118,2cyclictest32042-21awk07:15:270
2778799118,2cyclictest28839-21cut11:10:200
2778799118,2cyclictest28815-21cat11:30:150
2778799118,2cyclictest27419-21ssh12:08:020
2778799118,2cyclictest24682-21rm09:30:420
2778799118,2cyclictest23406-21cat12:25:150
2778799118,2cyclictest1903-21cat08:30:000
2778799118,2cyclictest18960-21tr09:05:150
2778799118,2cyclictest18011-21sed09:00:260
2778799118,2cyclictest17778-21grep07:55:150
2778799118,2cyclictest15854-21ssh10:05:050
2778799118,2cyclictest15082-21tr09:45:140
2778799118,2cyclictest14905-21ssh09:25:170
2778799118,2cyclictest1403-21ssh12:11:310
2778799118,2cyclictest13365-21smart_sda11:20:240
2778799118,2cyclictest10135-21wc08:45:190
2778799117,3cyclictest4295-21grep10:55:260
2778799114,5cyclictest7489-21/usr/sbin/munin12:35:160
169562110,0chrt0-21swapper/209:45:252
2779799108,1cyclictest28276-21cut07:10:153
2779799108,1cyclictest22752-21ssh12:25:013
2779799108,1cyclictest18947-21grep09:05:163
2779799108,1cyclictest18860-21sort07:55:263
2779799108,1cyclictest18627-21sed09:05:003
2779799108,1cyclictest168722sleep307:53:153
2779799108,1cyclictest12994-21cat08:50:243
2779799107,2cyclictest6994-21tr08:40:143
2779799107,2cyclictest21224-21ssh09:10:103
2779799100,9cyclictest30634-21expr07:15:153
2779499108,1cyclictest9074-21acpi08:45:102
2779499107,2cyclictest9826-21chrt07:40:002
2779499107,2cyclictest6730-21python07:30:232
2779499107,2cyclictest2943-21basename07:25:122
2779499107,2cyclictest23544-21cat08:05:252
2779499107,2cyclictest12884-21expr07:45:152
2779299108,1cyclictest158492sleep108:57:571
2779299108,1cyclictest1242-21grep07:20:161
2779299108,1cyclictest121872sleep107:44:471
2779299107,2cyclictest32131-21cut08:25:141
2779299107,2cyclictest29636-21ls08:20:151
2779299107,2cyclictest10853-21taskset08:45:301
2779299100,9cyclictest18418-21ntpq07:55:211
2778799108,1cyclictest31347-21taskset08:22:460
2778799108,1cyclictest31263-21ssh12:30:070
2778799108,1cyclictest15297-21ls07:50:140
2778799107,2cyclictest4980-21chrt07:29:010
2778799107,2cyclictest25541-21taskset08:10:210
2778799107,2cyclictest18321-21ssh10:45:130
2778799107,2cyclictest13896-21aten2_r7power_e08:55:120
2778799106,3cyclictest6615-21wc08:40:000
2778799105,2cyclictest225-21systemd-journal08:20:000
2778799100,9cyclictest2851-21cat08:30:180
277979995,3cyclictest11204-21cron08:50:003
277949997,1cyclictest162682sleep209:00:052
277929997,1cyclictest0-21swapper/108:44:101
277879997,1cyclictest1467-21if_err_enp0s31f07:20:170
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional