You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-12 - 14:47

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot7s.osadl.org (updated Sun Apr 12, 2026 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1875128774,9sleep20-21swapper/207:06:202
1870328664,6sleep10-21swapper/107:05:421
1884728371,8sleep00-21swapper/007:07:340
1882327865,9sleep30-21swapper/307:07:173
272312580,0sleep30-21swapper/312:10:423
1912299280,27cyclictest0-21swapper/110:00:371
1912599260,26cyclictest0-21swapper/209:58:162
1912599250,24cyclictest0-21swapper/209:15:032
19119992524,1cyclictest0-21swapper/011:09:070
1912599240,24cyclictest0-21swapper/211:49:552
1912299242,3cyclictest0-21swapper/111:35:191
1912299240,2cyclictest141rcu_preempt08:30:151
1912599230,23cyclictest0-21swapper/211:16:462
1912599230,23cyclictest0-21swapper/210:17:002
1912599230,20cyclictest0-21swapper/209:42:512
1912299231,1cyclictest141rcu_preempt12:09:491
19125992221,1cyclictest0-21swapper/210:36:232
19125992217,4cyclictest32-21ksoftirqd/211:50:152
19125992116,4cyclictest32-21ksoftirqd/211:02:142
1912299210,0cyclictest0-21swapper/110:25:121
19119992120,1cyclictest0-21swapper/011:17:210
19129992018,1cyclictest8943-21ssh11:21:333
1912599200,19cyclictest0-21swapper/209:39:582
19122992015,4cyclictest23141-21rm09:15:261
1912299200,1cyclictest0-21swapper/111:53:451
19125991910,1cyclictest141rcu_preempt10:11:112
1912599190,18cyclictest0-21swapper/210:42:172
1912599190,15cyclictest0-21swapper/212:12:252
1912299193,4cyclictest141rcu_preempt10:50:291
19119991918,1cyclictest13-21ksoftirqd/009:44:460
1912599180,18cyclictest0-21swapper/210:02:122
1912599180,0cyclictest141rcu_preempt11:44:222
1912299182,1cyclictest141rcu_preempt12:18:421
1912299182,1cyclictest12131-21diskmemload09:36:521
1912299180,2cyclictest0-21swapper/111:18:121
1911999180,18cyclictest0-21swapper/010:11:050
1912999170,2cyclictest0-21swapper/307:45:143
19125991714,2cyclictest0-21swapper/210:20:132
1912599170,1cyclictest0-21swapper/208:05:132
1912299176,9cyclictest537-21snmpd12:37:001
19122991717,0cyclictest0-21swapper/111:55:141
1912299171,2cyclictest12131-21diskmemload10:46:191
1912299170,1cyclictest0-21swapper/109:14:111
1912299170,14cyclictest0-21swapper/110:34:431
1911999171,15cyclictest0-21swapper/012:25:480
1911999170,17cyclictest0-21swapper/011:11:170
1911999170,14cyclictest0-21swapper/010:36:230
1912999160,1cyclictest0-21swapper/310:30:513
1912599163,12cyclictest0-21swapper/210:46:122
1912599162,0cyclictest141rcu_preempt09:46:362
19125991611,4cyclictest32-21ksoftirqd/211:07:142
1912299167,7cyclictest25-21ksoftirqd/111:00:151
1912299160,4cyclictest0-21swapper/110:10:201
1911999162,1cyclictest141rcu_preempt10:49:500
1911999160,4cyclictest0-21swapper/009:12:310
1912999159,4cyclictest30952-21sh09:40:033
1912999151,14cyclictest0-21swapper/310:15:133
19129991511,3cyclictest11516-21ssh10:25:243
1912999151,13cyclictest0-21swapper/310:56:123
1912599157,3cyclictest0-21swapper/212:00:242
1912599152,2cyclictest141rcu_preempt10:52:002
1912599150,14cyclictest0-21swapper/209:54:362
1912299150,2cyclictest0-21swapper/112:25:481
1912299150,15cyclictest0-21swapper/109:31:371
1911999150,7cyclictest0-21swapper/008:05:130
1911999150,1cyclictest0-21swapper/011:39:470
1911999150,13cyclictest0-21swapper/009:22:270
1912999143,6cyclictest0-21swapper/312:00:383
19129991413,1cyclictest0-21swapper/312:39:243
19129991410,3cyclictest13910-21ntpq11:05:173
19129991410,2cyclictest31419-21munin-run07:35:003
19129991410,2cyclictest14056-21ssh09:10:183
1912599149,4cyclictest32-21ksoftirqd/210:26:112
1912599145,4cyclictest27996-21apt-get10:55:132
1912599142,9cyclictest761-21/usr/sbin/munin12:15:162
19125991410,3cyclictest32-21ksoftirqd/207:50:122
19125991410,1cyclictest32-21ksoftirqd/211:28:392
19125991410,1cyclictest32-21ksoftirqd/211:28:392
19125991410,1cyclictest32-21ksoftirqd/209:13:222
1912599140,14cyclictest0-21swapper/211:13:482
1912599140,11cyclictest0-21swapper/212:34:002
1912599140,10cyclictest0-21swapper/208:45:132
19122991412,2cyclictest55750irq/128-enp2s0-10:38:041
1912299141,12cyclictest14788-21cron11:45:001
1912299140,2cyclictest0-21swapper/111:31:121
1912299140,12cyclictest22012-21grep07:15:131
1911999146,7cyclictest29050-21munin-run07:30:010
19119991412,2cyclictest13-21ksoftirqd/010:35:010
1911999141,11cyclictest0-21swapper/010:08:090
1912999139,3cyclictest935-21sed07:35:283
1912999139,3cyclictest25111-21grep07:20:173
1912999139,3cyclictest24483-21wc09:35:263
1912999139,3cyclictest13454-21kworker/3:210:43:423
1912999139,2cyclictest12675-21grep08:00:233
1912999133,2cyclictest225-21systemd-journal10:38:183
1912999131,2cyclictest0-21swapper/309:17:283
19129991310,2cyclictest4968-21cron11:20:003
19129991310,2cyclictest21876-21turbostat.cron11:30:003
19129991310,2cyclictest21876-21turbostat.cron11:30:003
19129991310,2cyclictest21736-21taskset09:53:373
19129991310,2cyclictest13454-21kworker/3:210:05:423
19129991310,2cyclictest13454-21kworker/3:209:46:223
1912599132,1cyclictest0-21swapper/212:37:022
1912599131,0cyclictest141rcu_preempt10:09:102
1912299139,3cyclictest24569-21grep08:25:211
1912299136,5cyclictest18570-21apt-get08:15:141
19122991312,1cyclictest29285-21ssh12:31:241
1912299131,1cyclictest141rcu_preempt09:57:551
1912299131,11cyclictest0-21swapper/112:01:411
19122991310,2cyclictest16788-21forks08:10:161
1911999139,1cyclictest13-21ksoftirqd/011:01:120
19119991310,2cyclictest3135-21grep08:50:140
19119991310,2cyclictest29229-21http09:20:000
19119991310,2cyclictest24786-21wc08:25:250
19119991310,2cyclictest1719-21grep12:15:170
19119991310,2cyclictest12119-21sh09:10:000
19119991310,2cyclictest10008-21grep10:25:130
1911999130,12cyclictest25180-21tr12:10:130
1911999130,12cyclictest0-21swapper/010:19:010
1911999130,11cyclictest0-21swapper/012:32:120
1912999129,2cyclictest8931-21ls12:20:003
1912999129,2cyclictest782-21tail10:00:163
1912999129,2cyclictest6996-21ssh09:25:193
1912999129,2cyclictest24614-21grep09:55:163
1912999129,2cyclictest20926-21ssh12:26:243
1912999125,3cyclictest31129-21cpuspeed_turbos11:35:163
1912999124,6cyclictest1717-21apt-get07:40:123
1912999120,2cyclictest0-21swapper/311:59:303
1912999120,11cyclictest23253-21ssh11:50:003
1912599129,2cyclictest5573-21ls07:45:262
1912599129,2cyclictest413-21ls07:35:232
1912599129,2cyclictest25000-21tail08:25:262
1912599129,2cyclictest20675-21cron08:20:002
1912599129,2cyclictest18452-21grep12:25:162
1912599129,2cyclictest10760-21grep09:05:182
1912599129,2cyclictest10335-21tr12:20:182
1912599128,3cyclictest18769-21sh09:32:182
1912599123,6cyclictest55550irq/126-enp2s0-12:05:162
1912599122,8cyclictest32-21ksoftirqd/208:30:152
19125991210,1cyclictest23680-21sed11:30:182
1912599120,4cyclictest0-21swapper/209:22:202
1912599120,11cyclictest522-21cut11:55:172
1912599120,11cyclictest1518-21ssh11:36:162
1912299129,2cyclictest24612-21tr07:20:141
1912299129,2cyclictest21004-21sed07:10:261
1912299129,2cyclictest14044-21grep11:05:191
1912299128,3cyclictest25-21ksoftirqd/111:49:251
1912299125,4cyclictest21145-21sh11:10:141
1912299122,1cyclictest12131-21diskmemload10:42:171
1912299120,11cyclictest5526-21grep08:55:141
1912299120,11cyclictest0-21swapper/109:25:251
1911999129,2cyclictest9292-21wc12:00:220
1911999129,2cyclictest858-21grep07:35:270
1911999129,2cyclictest7389-21basename09:00:000
1911999129,2cyclictest3282-21cat07:40:250
1911999129,2cyclictest32421-21grep11:55:140
1911999129,2cyclictest19507-21ssh11:28:110
1911999129,2cyclictest19507-21ssh11:28:110
1911999122,6cyclictest2597-21cstates12:35:170
1911999122,4cyclictest31110-21sshd07:31:160
1911999121,8cyclictest9605-21chrt11:22:020
1911999121,10cyclictest19622-21ssh10:50:130
1911999120,4cyclictest13486-21taskset08:04:190
1911999120,12cyclictest0-21swapper/009:47:460
1912999119,1cyclictest5259-21cut08:55:133
1912999118,2cyclictest697-21grep08:45:153
1912999118,2cyclictest5701-21ssh11:00:243
1912999118,2cyclictest4476-21grep08:50:253
1912999118,2cyclictest31133-21cut08:40:173
1912999118,2cyclictest29922-21ssh09:20:133
1912999118,2cyclictest28342-21grep08:35:153
1912999118,2cyclictest28342-21grep08:35:153
1912999118,2cyclictest28013-21ssh11:52:123
1912999118,2cyclictest25820-21wget08:30:163
1912999118,2cyclictest24679-21ssh11:30:403
1912999118,2cyclictest23461-21ssh11:10:433
1912999118,2cyclictest22611-21sed08:20:263
1912999118,2cyclictest22444-21tail07:15:153
1912999118,2cyclictest21366-21chrt07:13:583
1912999118,2cyclictest20672-21cut10:50:193
1912999118,2cyclictest19350-21grep08:15:163
1912999118,2cyclictest16767-21expr12:05:143
1912999118,2cyclictest16701-21ssh10:10:063
1912999118,2cyclictest16481-21tr08:10:153
1912999118,2cyclictest15750-21chrt08:07:553
1912999118,2cyclictest14288-21ssh12:22:463
1912999118,2cyclictest14273-21ssh09:30:093
1912999118,2cyclictest12887-21sed10:45:243
1912999118,2cyclictest10577-21ssh11:41:453
1912999118,2cyclictest10307-21wc07:55:203
1912999116,4cyclictest26889-21smartctl12:30:163
1912999116,2cyclictest225-21systemd-journal07:55:003
1912999115,5cyclictest2013-21ssh10:20:163
1912599118,2cyclictest9220-21sh09:26:252
1912599118,2cyclictest5534-21tr08:55:142
1912599118,2cyclictest29951-21cut07:30:152
1912599118,2cyclictest29036-21sh07:30:002
1912599118,2cyclictest25553-21cut07:20:212
1912599118,2cyclictest21929-21grep07:15:132
1912599118,2cyclictest10623-21sed07:55:242
1912599116,2cyclictest5517-21apt-get11:20:132
1912599110,2cyclictest0-21swapper/207:40:122
1912599110,1cyclictest0-21swapper/210:31:532
1912299119,1cyclictest7467-21tr09:45:141
1912299118,2cyclictest5616-21sh09:44:091
1912299118,2cyclictest4837-21grep07:45:181
1912299118,2cyclictest362-21sh10:57:371
1912299118,2cyclictest30154-21ssh10:17:391
1912299118,2cyclictest30088-21chrt08:38:471
1912299118,2cyclictest30088-21chrt08:38:461
1912299118,2cyclictest30057-21sed07:30:161
1912299118,2cyclictest2715-21wc08:50:001
1912299118,2cyclictest2483-21tail07:40:161
1912299118,2cyclictest1788-21ssh09:22:091
1912299118,2cyclictest14453-21ssh12:22:541
1912299118,2cyclictest10369-21tail10:05:251
1912299117,3cyclictest25-21ksoftirqd/107:35:131
1912299116,4cyclictest1624-21cpuspeed_turbos10:20:141
1912299116,2cyclictest7319-21grep07:50:171
1912299114,3cyclictest225-21systemd-journal09:04:361
1912299113,7cyclictest13859-21apache_accesses11:25:101
1912299113,7cyclictest13859-21apache_accesses11:25:101
1911999119,1cyclictest8421-21sed09:00:190
1911999119,1cyclictest8375-21sed07:50:260
1911999119,1cyclictest29632-21cat08:35:260
1911999119,1cyclictest29632-21cat08:35:260
1911999119,1cyclictest2804-21grep10:20:220
1911999119,1cyclictest24288-21sed11:30:230
1911999119,1cyclictest17250-21cat08:10:190
1911999118,2cyclictest6967-21sh11:40:130
1911999118,2cyclictest3102-21grep10:40:140
1911999118,2cyclictest28453-21expr10:55:140
1911999118,2cyclictest25677-21taskset08:30:050
1911999118,2cyclictest23991-21tr11:50:140
1911999118,2cyclictest23288-21ssh09:54:500
1911999118,2cyclictest21570-21ssh12:07:490
1911999118,2cyclictest20017-21cat08:15:240
1911999118,2cyclictest11251-21grep12:20:260
1911999115,3cyclictest21732-21apache_volume07:15:110
1911999112,7cyclictest17154-21unixbench-2d11:45:250
1911999111,0cyclictest141rcu_preempt09:25:020
1912999108,1cyclictest11537-21sed09:05:273
1912999107,2cyclictest9544-21chrt09:02:323
1912999107,2cyclictest23608-21tr08:25:153
1912599108,2cyclictest20278-21grep07:10:202
1912599108,1cyclictest94942sleep209:01:552
1912599108,1cyclictest114002sleep208:00:052
1912599100,2cyclictest0-21swapper/208:40:132
1912299108,1cyclictest9249-21grep07:55:131
1912299108,1cyclictest30553-21wget08:40:141
1912299108,1cyclictest2873-21chrt08:50:121
1912299108,1cyclictest286722sleep107:25:271
1912299108,1cyclictest26141-21ntp_kernel_pll_12:10:191
1912299108,1cyclictest229152sleep108:23:041
1912299108,1cyclictest16015-21expr09:50:141
1912299108,1cyclictest14500-21grep08:05:171
1912299108,1cyclictest100472sleep109:05:151
1912299107,2cyclictest7554-21ssh11:20:271
1912299107,2cyclictest13554-21ls08:05:001
1911999108,1cyclictest3949-21ssh10:01:520
1911999108,1cyclictest3482sleep008:45:020
1911999108,1cyclictest23429-21rm09:35:170
1911999108,1cyclictest21405-21cat08:20:150
1911999108,1cyclictest19391-21grep07:10:130
1911999108,1cyclictest16514-21ssh09:30:320
1911999108,1cyclictest11215-21cat08:00:000
1911999107,2cyclictest24834-21expr09:55:160
191299993,3cyclictest225-21systemd-journal07:25:023
191259997,1cyclictest4325-21sed08:50:252
191259996,2cyclictest28810-21grep08:35:172
191259996,2cyclictest28810-21grep08:35:172
191259996,2cyclictest18413-21sort08:15:002
191259990,2cyclictest0-21swapper/208:20:152
191199998,1cyclictest4411-21expr07:45:150
191199992,4cyclictest26302-21sshd07:23:160
191199990,1cyclictest0-21swapper/008:40:130
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional