You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-03 - 00:22

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot7s.osadl.org (updated Tue Jun 02, 2026 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
18916210349,48sleep20-21swapper/207:08:542
1886729961,33sleep00-21swapper/007:08:150
1866929068,6sleep10-21swapper/107:05:401
1867128369,9sleep30-21swapper/307:05:413
1909599280,0cyclictest0-21swapper/310:26:393
1908599270,27cyclictest0-21swapper/009:46:120
19095992625,1cyclictest39-21ksoftirqd/312:27:263
1909599250,24cyclictest0-21swapper/310:03:333
1909599240,0cyclictest0-21swapper/310:21:483
1909599240,0cyclictest0-21swapper/310:17:093
19095992322,1cyclictest0-21swapper/311:59:063
1909599230,1cyclictest0-21swapper/311:11:113
1909499230,20cyclictest0-21swapper/210:55:342
1908999230,22cyclictest0-21swapper/110:17:091
19085992322,1cyclictest0-21swapper/012:18:020
1909599221,2cyclictest0-21swapper/309:34:163
1908999221,20cyclictest1354-21ssh10:40:191
1908999220,3cyclictest0-21swapper/109:32:101
1908999220,18cyclictest0-21swapper/111:58:501
19095992119,2cyclictest39-21ksoftirqd/312:01:333
19094992116,5cyclictest32126-21kworker/u8:209:55:292
19094992113,4cyclictest225-21systemd-journal07:45:002
19089992120,1cyclictest0-21swapper/112:07:441
1908999210,21cyclictest0-21swapper/110:26:391
1908599215,8cyclictest0-21swapper/010:15:140
1908599213,14cyclictest30020-21ssh09:40:130
1908599213,14cyclictest30020-21ssh09:40:130
1909599200,2cyclictest0-21swapper/312:36:353
1908999202,1cyclictest12105-21diskmemload11:02:121
1908999202,1cyclictest0-21swapper/111:41:041
1908999200,19cyclictest0-21swapper/109:41:461
1908999200,19cyclictest0-21swapper/109:41:461
1909599190,18cyclictest0-21swapper/311:02:393
1909499191,17cyclictest0-21swapper/212:39:372
1908999190,1cyclictest0-21swapper/110:03:331
1908999190,17cyclictest0-21swapper/110:38:401
1909599180,0cyclictest0-21swapper/310:09:363
19094991816,2cyclictest0-21swapper/210:17:212
1909499181,1cyclictest0-21swapper/210:00:222
1909499180,2cyclictest0-21swapper/211:52:122
1909499180,17cyclictest0-21swapper/207:55:142
1908999180,1cyclictest0-21swapper/112:02:511
1908999180,1cyclictest0-21swapper/111:51:491
1908999180,17cyclictest0-21swapper/109:57:001
1908999180,15cyclictest0-21swapper/110:47:011
1908599180,18cyclictest0-21swapper/012:07:540
1909599171,0cyclictest0-21swapper/311:36:433
1908999170,1cyclictest0-21swapper/112:30:001
1908999170,1cyclictest0-21swapper/110:21:481
19085991716,1cyclictest0-21swapper/010:53:310
1909599162,11cyclictest12105-21diskmemload11:21:093
1909599161,2cyclictest0-21swapper/311:51:473
19094991613,2cyclictest10550irq/122-ahci[0011:40:002
19094991613,2cyclictest0-21swapper/211:40:132
19094991612,3cyclictest32-21ksoftirqd/209:33:172
1909499160,1cyclictest0-21swapper/210:28:322
1909499160,16cyclictest0-21swapper/210:34:322
1909499160,15cyclictest0-21swapper/212:02:402
19089991615,1cyclictest0-21swapper/112:30:091
19089991615,1cyclictest0-21swapper/111:36:011
19089991610,5cyclictest25-21ksoftirqd/108:20:001
1908999160,16cyclictest0-21swapper/111:11:111
19085991615,1cyclictest13-21ksoftirqd/012:00:130
19095991512,3cyclictest39-21ksoftirqd/312:23:023
1909599150,2cyclictest0-21swapper/309:29:583
19094991514,1cyclictest32-21ksoftirqd/209:49:272
1909499151,1cyclictest11211-21kworker/u8:012:07:282
1909499150,7cyclictest30058-21sensors11:55:242
1908999157,7cyclictest23190-21awk12:10:281
1908999150,15cyclictest0-21swapper/109:27:521
1908599159,4cyclictest6222-21apt-get07:50:140
1909599143,9cyclictest6644-21fschecks_count09:45:163
1909599142,9cyclictest11075-21ssh11:05:233
1909599140,2cyclictest0-21swapper/310:37:323
1909599140,1cyclictest0-21swapper/309:24:003
19094991411,2cyclictest5625-21tail09:25:172
19094991411,2cyclictest4233-21wc11:20:262
1909499141,12cyclictest25954-21ls11:15:002
19094991410,3cyclictest10417-21sort11:05:182
1909499140,4cyclictest0-21swapper/209:10:132
1909499140,14cyclictest0-21swapper/209:24:502
1908999144,7cyclictest12637-21apt-get09:10:141
19089991411,2cyclictest0-21swapper/112:15:301
19085991413,1cyclictest0-21swapper/010:22:260
19085991411,2cyclictest20398-21grep11:50:140
19085991410,4cyclictest0-21swapper/010:44:460
1909599131,2cyclictest0-21swapper/312:14:293
19095991312,1cyclictest0-21swapper/311:32:243
1909599130,12cyclictest0-21swapper/309:10:483
1909499139,3cyclictest21036-21sed11:30:262
1909499138,4cyclictest32-21ksoftirqd/208:25:142
19094991311,1cyclictest32-21ksoftirqd/211:16:112
19094991310,2cyclictest9218-21perf11:05:002
19094991310,2cyclictest8958-21ssh10:45:152
19094991310,2cyclictest30950-21cron07:35:002
19094991310,2cyclictest24883-21sleep10:55:002
19094991310,2cyclictest23140-21wc09:35:212
19094991310,2cyclictest21496-21sed08:20:142
19094991310,2cyclictest19613-21sh11:50:002
19094991310,2cyclictest11718-21grep08:00:172
1909499130,12cyclictest22975-21awk09:15:282
1908999139,4cyclictest23332-21taskset09:15:451
1908999132,11cyclictest15637-21ssh11:46:541
19089991312,1cyclictest0-21swapper/111:09:421
19089991312,1cyclictest0-21swapper/108:10:231
19089991310,2cyclictest5925-21cut07:50:001
19089991310,2cyclictest24329-21tr07:20:141
19089991310,2cyclictest15862-21cat09:50:231
1908999130,8cyclictest0-21swapper/111:16:041
1908999130,3cyclictest0-21swapper/112:36:351
1908999130,13cyclictest0-21swapper/111:26:251
1908599139,3cyclictest15482-21munin-run08:10:000
1908599138,4cyclictest13-21ksoftirqd/011:15:130
19085991311,2cyclictest13-21ksoftirqd/012:25:340
19085991311,1cyclictest25098-1kworker/u9:011:30:160
19085991310,2cyclictest22873-21chrt10:34:150
19085991310,2cyclictest2104-21ssh11:00:160
19085991310,2cyclictest15399-21grep09:30:260
19085991310,2cyclictest1031-21wc07:40:000
1908599130,13cyclictest0-21swapper/009:23:350
1909599129,2cyclictest5188-21sed07:45:253
1909599129,2cyclictest32563-21cron08:45:003
1909599129,2cyclictest31065-21ssh09:40:193
1909599129,2cyclictest31065-21ssh09:40:193
1909599129,2cyclictest24872-21ls10:55:003
1909599129,2cyclictest21708-21ssh09:35:123
1909599129,2cyclictest17841-21ssh09:51:433
1909599127,4cyclictest1142-21runrttasks10:32:093
1909599126,1cyclictest12105-21diskmemload12:05:453
1909599122,8cyclictest9996-21apt-get09:05:123
1909599122,8cyclictest3651-21cpuspeed_turbos11:40:143
1909599122,4cyclictest225-21systemd-journal07:35:003
19095991210,1cyclictest26370-21cut08:30:163
19095991210,1cyclictest26217-21grep10:55:173
1909599120,5cyclictest0-21swapper/311:47:033
1909599120,3cyclictest19593-21cpuspeed_turbos07:10:153
1909599120,2cyclictest25051-21sh09:55:513
1909599120,1cyclictest0-21swapper/311:25:133
1909599120,11cyclictest26345-21ssh11:15:023
1909499129,3cyclictest55850irq/129-enp2s0-12:27:472
1909499129,2cyclictest6955-21tail07:50:172
1909499129,2cyclictest32256-21sh10:40:002
1909499129,2cyclictest3043-21tr08:50:152
1909499129,2cyclictest29635-21cut12:15:132
1909499129,2cyclictest29264-21sed08:35:212
1909499129,2cyclictest22407-21grep07:15:162
1909499129,2cyclictest21714-21sed12:10:162
1909499129,2cyclictest1782-21wc10:40:222
1909499129,2cyclictest14588-21cron10:10:002
1909499129,2cyclictest10617-21sort09:05:152
1909499128,3cyclictest23381-21wc12:30:222
1909499128,3cyclictest14467-21grep08:05:192
1909499128,2cyclictest5196-21sed07:45:252
1909499128,2cyclictest24166-21grep07:20:142
1909499128,2cyclictest11149-21ssh11:25:142
1908999129,2cyclictest7306-21grep07:50:201
1908999129,2cyclictest6192-21cut12:20:161
1908999129,2cyclictest3415-21if_err_enp0s31f11:20:191
1908999129,2cyclictest28234-21rm10:56:061
1908999129,2cyclictest23508-21ssh10:53:561
1908999129,2cyclictest23060-21cat08:25:011
1908999129,2cyclictest15761-21fschecks_count10:10:171
1908999129,2cyclictest1127-21grep08:45:181
1908999129,2cyclictest11067-21cat09:05:191
19089991210,1cyclictest2768-21cat07:40:271
1908999120,1cyclictest0-21swapper/110:31:591
1908599129,2cyclictest31696-21sed10:00:180
1908599129,2cyclictest23047-21ssh09:55:150
1908599129,2cyclictest13-21ksoftirqd/011:35:520
1908599129,2cyclictest12954-21wc09:30:000
1908599129,2cyclictest12455-21cut11:45:160
1908599129,2cyclictest11912-21cat11:25:190
1908599129,2cyclictest11599-21grep09:05:250
1908599129,2cyclictest11170-21head11:05:240
1908599124,2cyclictest225-21systemd-journal07:25:000
19085991210,1cyclictest29880-21cat07:30:200
19085991210,1cyclictest12105-21diskmemload12:32:330
1908599120,2cyclictest0-21swapper/011:12:350
1909599119,1cyclictest31294-21grep07:35:143
1909599119,1cyclictest2867-21aten2_r7power_p08:50:143
1909599119,1cyclictest21201-21tr09:15:153
1909599119,1cyclictest19986-21grep08:15:253
1909599119,1cyclictest16591-21cut08:10:153
1909599119,1cyclictest11277-21tr08:00:143
1909599118,2cyclictest8326-21tail09:00:173
1909599118,2cyclictest6912-21cut07:50:173
1909599118,2cyclictest32733-21aten2_r7power_c10:40:133
1909599118,2cyclictest27206-21tail07:25:173
1909599118,2cyclictest25723-21sed07:20:263
1909599118,2cyclictest24411-21grep08:25:213
1909599118,2cyclictest23803-21cron07:20:003
1909599118,2cyclictest22211-21ssh12:30:133
1909599118,2cyclictest20963-21tr08:20:123
1909599118,2cyclictest16882-21wc10:10:253
1909599117,3cyclictest31784-21ssh12:15:383
1909599111,7cyclictest32460-21kworker/u8:210:45:143
1909499119,1cyclictest9075-21cat09:00:262
1909499119,1cyclictest15409-21tr10:10:142
1909499118,2cyclictest649-21sh09:41:162
1909499118,2cyclictest649-21sh09:41:162
1909499118,2cyclictest640-21tr08:45:152
1909499118,2cyclictest26717-21tr07:25:142
1909499118,2cyclictest18549-21tr08:15:142
1909499118,2cyclictest14859-21cut09:50:142
1909499118,2cyclictest1437-21ssh10:20:312
1909499117,3cyclictest32283-21wc07:35:202
1908999119,2cyclictest32754-21sh09:21:511
1908999119,2cyclictest12943-21taskset08:03:121
1908999119,2cyclictest11230-21ssh10:07:201
1908999119,1cyclictest9190-21ls09:00:261
1908999119,1cyclictest31282-21basename08:40:181
1908999119,1cyclictest28161-21aten2_r7power_p08:35:141
1908999119,1cyclictest25917-21grep08:30:141
1908999119,1cyclictest18969-21ssh11:30:051
1908999118,2cyclictest5838-21grep08:55:181
1908999118,2cyclictest21675-21basename09:35:111
1908999118,2cyclictest13848-21taskset08:05:151
1908999116,3cyclictest6650-21fschecks_count09:45:171
1908999115,6cyclictest6028-21apache207:35:211
1908999114,4cyclictest10706-21idleruntime-cro08:00:001
1908599119,1cyclictest26635-21cut10:55:200
1908599118,2cyclictest8684-21basename07:55:150
1908599118,2cyclictest7970-21expr09:00:150
1908599118,2cyclictest6378-21cut12:40:000
1908599118,2cyclictest4528-21sh11:40:200
1908599118,2cyclictest4419-21sed11:20:270
1908599118,2cyclictest27617-21ntpq07:25:200
1908599118,2cyclictest27453-21ssh09:19:150
1908599118,2cyclictest25419-21cron08:29:590
1908599118,2cyclictest22412-21grep07:15:160
1908599118,2cyclictest21702-21ssh09:35:120
1908599118,2cyclictest21410-21ssh10:13:490
1908599118,2cyclictest17141-21ssh09:51:100
1908599118,2cyclictest16531-21perf10:50:000
1908599118,2cyclictest12759-21ssh12:24:330
1908599113,4cyclictest225-21systemd-journal07:40:280
1908599110,1cyclictest0-21swapper/009:10:140
1908599110,10cyclictest26763-21ssh10:35:450
1909599108,1cyclictest29414-21basename08:35:233
1909599107,2cyclictest5435-21tr08:55:153
1909599107,2cyclictest2631-21grep08:50:003
1909599107,2cyclictest2019-21basename07:40:193
1909599107,2cyclictest13989-21sort08:05:163
1909599105,4cyclictest8562-21/usr/sbin/munin07:55:133
1909499108,1cyclictest6968-21taskset08:55:302
1909499108,1cyclictest5251-21sh12:20:042
1909499108,1cyclictest324032sleep208:43:012
1909499108,1cyclictest0-21swapper/207:10:142
1909499107,2cyclictest27237-21wc08:30:252
1909499102,4cyclictest225-21systemd-journal08:13:022
1908999108,2cyclictest4630-21taskset08:51:441
1908999108,1cyclictest26861-21expr07:25:141
1908999108,1cyclictest22450-21tail07:15:171
1908999108,1cyclictest19590-21expr07:10:151
1908599108,1cyclictest3842-21grep07:45:130
1908599108,1cyclictest2926-21wget08:50:150
1908599108,1cyclictest26915-21grep08:30:210
1908599108,1cyclictest21000-21cut08:20:120
1908599107,2cyclictest9476-21ssh10:05:580
1908599107,2cyclictest672-21cut08:45:150
1908599107,2cyclictest5274-21aten2_r7power_v08:55:150
1908599107,2cyclictest32397-21ssh11:56:560
1908599107,2cyclictest28218-21grep08:35:140
1908599107,2cyclictest20966-21ssh12:10:040
1908599107,2cyclictest19944-21fschecks_count07:10:170
1908599107,2cyclictest11186-21ssh10:26:360
1908599107,2cyclictest10991-21apt08:00:120
1908599106,3cyclictest30602-21tr08:40:140
1908599104,5cyclictest17548-21unixbench-2d08:10:230
190899998,1cyclictest30772-21chrt07:32:441
190899997,1cyclictest23270-21test08:25:021
190859997,1cyclictest18413-21grep08:15:130
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional