You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-23 - 23:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot7s.osadl.org (updated Sun Aug 23, 2026 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
136029380,9sleep00-21swapper/007:09:300
133329062,16sleep10-21swapper/107:09:121
108328977,8sleep20-21swapper/207:06:042
107428859,9sleep30-21swapper/307:05:573
189312730,0sleep10-21swapper/109:24:021
282822560,0sleep10-21swapper/112:03:231
148699380,1cyclictest0-21swapper/207:15:152
1477992724,2cyclictest13-21ksoftirqd/009:49:020
1486992417,6cyclictest32-21ksoftirqd/209:43:342
148699241,2cyclictest55550irq/126-enp2s0-09:50:232
148299240,1cyclictest0-21swapper/109:49:571
148299230,23cyclictest0-21swapper/110:49:491
1477992322,1cyclictest0-21swapper/011:46:330
1486992218,4cyclictest0-21swapper/209:10:232
148299221,1cyclictest27082-21diskmemload10:58:451
1490992120,1cyclictest0-21swapper/310:50:533
1477992120,1cyclictest0-21swapper/011:51:550
1486992017,2cyclictest55750irq/128-enp2s0-09:36:332
1486992014,5cyclictest27082-21diskmemload11:15:232
148699201,18cyclictest0-21swapper/212:01:132
149099190,2cyclictest0-21swapper/311:26:413
149099190,18cyclictest13307-21ssh11:54:523
147799192,8cyclictest0-21swapper/008:40:140
147799190,2cyclictest0-21swapper/009:17:410
1486991816,1cyclictest0-21swapper/211:10:032
148299180,3cyclictest0-21swapper/111:24:551
148299180,18cyclictest0-21swapper/112:30:351
1486991715,1cyclictest55550irq/126-enp2s0-10:46:032
148699171,1cyclictest0-21swapper/211:21:592
1482991715,1cyclictest27082-21diskmemload11:40:561
148299171,1cyclictest27082-21diskmemload12:29:091
148299170,2cyclictest0-21swapper/110:50:151
1477991715,1cyclictest0-21swapper/012:00:140
147799171,2cyclictest0-21swapper/011:30:130
147799170,15cyclictest0-21swapper/010:49:280
149099160,0cyclictest0-21swapper/311:36:173
1486991615,1cyclictest0-21swapper/212:13:032
1486991612,4cyclictest0-21swapper/211:48:432
1482991614,2cyclictest0-21swapper/110:12:331
1486991514,1cyclictest55850irq/129-enp2s0-10:07:532
1486991514,1cyclictest0-21swapper/211:50:252
1486991512,2cyclictest32-21ksoftirqd/211:40:212
1486991511,4cyclictest0-21swapper/209:48:042
148699150,15cyclictest0-21swapper/212:39:222
147799158,7cyclictest0-21swapper/010:35:260
1477991511,2cyclictest4626-21cron09:35:000
1477991510,3cyclictest20437-21cron11:40:000
149099145,6cyclictest24067-21ssh12:20:133
1490991413,1cyclictest0-21swapper/310:30:053
149099140,2cyclictest0-21swapper/312:06:013
149099140,13cyclictest0-21swapper/309:54:553
148699149,3cyclictest18677-21grep11:00:152
148699142,10cyclictest3615-21grep08:20:142
1486991413,1cyclictest0-21swapper/211:30:462
1486991412,2cyclictest0-21swapper/209:19:342
1486991410,2cyclictest12423-21grep09:20:132
148699140,13cyclictest0-21swapper/210:55:202
148299141,1cyclictest27082-21diskmemload10:08:121
148299141,1cyclictest13751-21kworker/u8:211:18:001
148299141,12cyclictest27082-21diskmemload09:43:471
1482991411,2cyclictest19730-21chrt10:21:361
148299141,12cyclictest12297-21cut07:30:181
148299140,14cyclictest0-21swapper/110:41:381
147799142,6cyclictest10840-21munin-run11:15:000
1477991410,2cyclictest18848-21grep07:45:160
147799140,14cyclictest0-21swapper/009:29:170
147799140,14cyclictest0-21swapper/009:29:160
149099139,2cyclictest1471-21cyclictest11:11:063
149099133,9cyclictest0-21swapper/310:19:593
1490991310,2cyclictest3820-21cron07:15:003
1490991310,2cyclictest32274-21wc08:10:253
1490991310,2cyclictest30465-21head11:45:203
1490991310,2cyclictest27665-21grep11:05:173
1490991310,2cyclictest21968-21tr09:25:213
1490991310,2cyclictest21968-21tr09:25:203
1490991310,2cyclictest13683-21ls09:20:213
1490991310,2cyclictest1156-21head10:10:273
149099130,12cyclictest30846-21ssh09:30:263
149099130,12cyclictest21126-21cut11:40:143
148699139,3cyclictest24687-21tr07:55:232
148699139,2cyclictest28617-21tr08:05:162
148699138,4cyclictest25242-21apt10:45:002
1486991313,0cyclictest0-21swapper/212:23:532
1486991312,1cyclictest2158-21ssh10:30:232
1486991312,1cyclictest0-21swapper/210:17:532
1486991310,3cyclictest0-21swapper/210:28:432
1486991310,2cyclictest27082-21diskmemload12:19:132
1486991310,2cyclictest22707-21sh10:05:002
148699130,2cyclictest0-21swapper/210:24:332
148299139,2cyclictest11976-21wc09:20:011
1482991310,2cyclictest27096-21grep08:00:201
1482991310,2cyclictest16488-21wc08:45:221
1482991310,2cyclictest15289-21grep11:55:191
1482991310,2cyclictest1527-21grep10:30:171
1482991310,2cyclictest13387-21sed11:35:161
148299130,3cyclictest0-21swapper/112:38:401
147799137,4cyclictest28477-21apache209:50:540
1477991310,2cyclictest431-21ssh10:10:200
1477991310,2cyclictest4079-21head08:20:170
1477991310,2cyclictest23960-21grep10:05:170
1477991310,2cyclictest18384-21head08:50:160
1477991310,2cyclictest11426-21tr10:55:200
147799130,13cyclictest0-21swapper/012:16:510
147799130,13cyclictest0-21swapper/011:15:460
149099129,2cyclictest9313-21head12:30:193
149099129,2cyclictest8326-21grep08:30:153
149099129,2cyclictest5358-21grep09:15:253
149099129,2cyclictest5020-21grep07:15:183
149099129,2cyclictest488-21ssh12:25:153
149099129,2cyclictest27629-21sed10:45:263
149099129,2cyclictest14534-21grep07:35:153
149099128,3cyclictest7879-21ssh09:36:003
149099128,3cyclictest18543-21sh07:45:013
149099127,3cyclictest15982-21cut08:45:173
149099126,2cyclictest225-21systemd-journal07:55:013
149099121,11cyclictest0-21swapper/311:34:463
1490991211,1cyclictest0-21swapper/309:49:253
148699129,2cyclictest8810-21sh07:25:002
148699129,2cyclictest8737-21ssh10:35:032
148699129,2cyclictest6421-21head08:25:162
148699129,2cyclictest31741-21tr10:10:142
148699129,2cyclictest2424-21grep08:15:292
148699129,2cyclictest17083-21grep07:40:162
148699129,2cyclictest13588-21grep08:40:172
148699129,2cyclictest11918-21grep07:30:172
148699128,3cyclictest28248-21rm11:05:212
148699128,2cyclictest14422-21grep11:35:262
148699121,11cyclictest0-21swapper/208:58:592
148699120,11cyclictest15829-21wc11:55:242
148299129,2cyclictest9921-21sort07:25:161
148299129,2cyclictest8948-21head12:10:261
148299129,2cyclictest8156-21grep10:15:151
148299129,2cyclictest5202-21tr09:35:151
148299129,2cyclictest402-21cron08:14:591
148299129,2cyclictest3290-21cut11:10:151
148299129,2cyclictest28974-21ntpq09:10:211
148299129,2cyclictest27068-21sh09:10:011
148299129,2cyclictest23148-21grep09:00:161
148299129,2cyclictest22080-21grep09:25:221
148299129,2cyclictest22080-21grep09:25:211
148299129,2cyclictest13958-21cut08:40:201
148299129,2cyclictest12601-21basename08:39:591
148299129,2cyclictest10378-21wc08:35:001
148299128,3cyclictest16626-21grep12:15:211
148299120,11cyclictest2359-21ssh12:06:501
147799129,2cyclictest9413-21mailstats10:15:230
147799129,2cyclictest8644-21wc12:10:250
147799129,2cyclictest6559-21grep09:35:230
147799129,2cyclictest2934-21ls07:10:210
147799129,2cyclictest24694-21wc09:05:000
147799129,2cyclictest24336-21head07:55:190
147799129,2cyclictest24188-21cat10:25:000
147799129,2cyclictest15849-21grep08:45:170
147799129,2cyclictest15279-21wc09:40:240
147799129,2cyclictest1522-21cut10:30:170
147799129,2cyclictest10520-21grep07:25:210
147799129,2cyclictest0-21swapper/009:21:500
147799128,3cyclictest18373-21sensors12:35:230
147799128,2cyclictest6457-21ssh09:55:120
149099119,1cyclictest13641-21chrt07:33:263
149099118,2cyclictest9875-21ssh12:11:083
149099118,2cyclictest9567-21tr07:25:143
149099118,2cyclictest9368-21ssh10:35:163
149099118,2cyclictest6232-21ssh09:55:043
149099118,2cyclictest31254-21ssh09:11:393
149099118,2cyclictest30133-21sh10:09:133
149099118,2cyclictest26221-21ssh12:01:473
149099118,2cyclictest26046-21cut08:00:143
149099118,2cyclictest25899-21ntp_offset10:25:203
149099118,2cyclictest25218-21grep07:55:283
149099118,2cyclictest22885-21sh12:38:393
149099118,2cyclictest21768-21chrt08:56:113
149099118,2cyclictest21705-21ssh10:42:133
149099118,2cyclictest19610-21cut11:00:203
149099118,2cyclictest17923-21ssh10:59:573
149099118,2cyclictest16690-21sh12:15:223
149099118,2cyclictest14543-21df11:55:143
149099118,2cyclictest13756-21cat09:40:133
149099118,2cyclictest11538-21grep11:15:163
149099110,5cyclictest0-21swapper/310:20:153
149099110,10cyclictest5083-21chrt08:20:293
149099110,10cyclictest23141-21ssh11:21:323
148699118,3cyclictest0-21swapper/209:25:242
148699118,3cyclictest0-21swapper/209:25:232
148699118,2cyclictest9872-21df07:25:152
148699118,2cyclictest342-21sh12:05:272
148699118,2cyclictest30454-21taskset11:25:402
148699118,2cyclictest30043-21sort09:30:172
148699118,2cyclictest26710-21head08:00:172
148699118,2cyclictest21049-21date07:50:002
148699118,2cyclictest19207-21sed08:50:272
148699118,2cyclictest17196-21chrt08:48:252
148699118,2cyclictest16146-21sh07:40:002
148699118,2cyclictest1603-21ssh12:25:232
148699118,2cyclictest10260-21ssh09:56:512
148699117,3cyclictest9256-21tail12:30:182
148699117,2cyclictest3814-21ssh10:50:282
148699117,2cyclictest10185-21cron08:34:592
148699113,4cyclictest3827-21turbostat.cron07:15:002
148299118,2cyclictest7751-21ssh11:31:391
148299118,2cyclictest7393-21grep08:25:251
148299118,2cyclictest30950-21ssh11:26:031
148299118,2cyclictest30861-21cat11:45:211
148299118,2cyclictest30274-21ssh09:50:021
148299118,2cyclictest29102-21head08:05:171
148299118,2cyclictest2863-21basename08:20:001
148299118,2cyclictest28293-21ssh10:26:461
148299118,2cyclictest25248-21sed07:55:291
148299118,2cyclictest23690-21sh12:20:001
148299118,2cyclictest2038-21ssh09:32:591
148299118,2cyclictest19422-21ssh11:00:181
148299118,2cyclictest15151-21tr10:00:141
148299118,2cyclictest11683-21ssh11:53:371
148299117,3cyclictest4989-21grep08:20:261
148299110,2cyclictest27082-21diskmemload11:08:251
147799119,1cyclictest28174-21cat08:05:000
147799118,2cyclictest32444-21ssh11:27:130
147799118,2cyclictest32230-21grep12:05:210
147799118,2cyclictest28782-21ls08:05:150
147799118,2cyclictest28595-21ssh11:05:240
147799118,2cyclictest25260-21cat09:05:160
147799118,2cyclictest25200-21ssh11:04:050
147799118,2cyclictest24850-21tr10:25:160
147799118,2cyclictest24443-21ssh10:44:230
147799118,2cyclictest24022-21aten2_r7power_c12:20:130
147799118,2cyclictest22068-21ssh11:20:420
147799118,2cyclictest17059-21grep07:40:150
147799118,2cyclictest16944-21ssh10:00:310
147799118,2cyclictest16377-21cat12:35:000
147799118,2cyclictest1536-21ssh09:13:450
147799118,2cyclictest1505-21ssh10:50:020
147799118,2cyclictest12778-21cut07:30:220
147799118,2cyclictest12129-21sed08:35:250
147799117,3cyclictest31541-21sed08:10:160
147799113,4cyclictest225-21systemd-journal11:57:570
147799110,10cyclictest3445-21ssh12:26:340
147799110,10cyclictest22989-21ssh11:40:280
149099108,1cyclictest29339-21cut08:05:203
149099108,1cyclictest22319-21taskset09:00:023
149099108,1cyclictest2220-21cat08:15:283
149099108,1cyclictest19172-21expr07:45:163
149099108,1cyclictest17555-21taskset08:50:043
149099108,1cyclictest14531-21unixbench_singl08:40:263
149099108,1cyclictest12546-21cat08:39:553
149099102,3cyclictest14831-21ssh10:00:113
149099101,8cyclictest0-21swapper/307:20:133
149099100,9cyclictest5856-21egrep08:25:133
148699108,1cyclictest380-21chrt08:14:472
148699108,1cyclictest21928-21tr07:50:172
148699107,2cyclictest24004-21sed09:00:262
148699100,9cyclictest26410-21cat09:05:262
148299108,1cyclictest7426-21if_err_enp0s31f09:55:181
148299108,1cyclictest7008-21ls07:20:171
148299108,1cyclictest3042-21expr07:10:221
148299108,1cyclictest17870-21cut08:50:151
148299107,2cyclictest6385-21chrt07:19:401
148299107,2cyclictest22322-21ntpq07:50:201
148299107,2cyclictest19935-21tr07:45:221
148299107,2cyclictest16430-21apache_accesses07:40:101
148299107,2cyclictest15597-21sed07:35:261
148299103,3cyclictest225-21systemd-journal10:35:011
147799108,1cyclictest5677-21taskset08:25:090
147799108,1cyclictest19952-21cat08:55:110
147799108,1cyclictest15691-21grep07:35:260
147799108,1cyclictest100132sleep008:32:530
147799107,2cyclictest6413-21cron07:20:000
147799107,2cyclictest23313-21basename07:55:000
147799100,4cyclictest8827-21idleruntime-cro07:25:010
14909997,1cyclictest25425-21cat09:05:153
14869996,2cyclictest10419-21chrt08:35:062
14829997,1cyclictest20084-21aten2_r7power_v08:55:141
14779997,2cyclictest1867-21ntpq08:15:220
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional