You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-02 - 01:49

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot7s.osadl.org (updated Mon Jun 01, 2026 12:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
482229482,8sleep00-21swapper/007:09:240
485128977,8sleep20-21swapper/207:09:452
478328965,6sleep10-21swapper/107:08:521
467028766,17sleep30-21swapper/307:07:253
4959992524,1cyclictest0-21swapper/110:52:311
495599250,1cyclictest0-21swapper/010:49:430
495999220,22cyclictest0-21swapper/111:43:491
496499210,1cyclictest0-21swapper/311:38:253
4959992118,2cyclictest0-21swapper/111:55:011
495999210,20cyclictest0-21swapper/110:30:131
496499201,1cyclictest0-21swapper/312:25:053
495999200,2cyclictest0-21swapper/112:19:351
4955992017,2cyclictest13-21ksoftirqd/011:11:220
496199196,12cyclictest0-21swapper/211:41:432
495999192,1cyclictest30015-21diskmemload11:11:101
4959991919,0cyclictest0-21swapper/111:01:201
495999190,18cyclictest0-21swapper/110:47:371
495999190,18cyclictest0-21swapper/110:39:341
495599190,2cyclictest0-21swapper/009:30:300
496499180,17cyclictest0-21swapper/310:25:143
496199180,2cyclictest24393-21turbostat.cron12:24:552
4959991813,3cyclictest25988-21munin-run10:25:001
495999181,17cyclictest30015-21diskmemload11:24:401
495999180,17cyclictest0-21swapper/112:21:571
495999180,17cyclictest0-21swapper/109:51:091
4964991715,1cyclictest0-21swapper/308:15:273
496199170,17cyclictest0-21swapper/210:42:222
496199170,15cyclictest0-21swapper/212:12:192
4959991710,4cyclictest30015-21diskmemload11:50:131
4959991710,4cyclictest30015-21diskmemload11:50:121
495599175,7cyclictest0-21swapper/010:10:170
495599175,7cyclictest0-21swapper/010:10:170
495599171,11cyclictest25543-21grep09:45:190
495599170,0cyclictest0-21swapper/010:34:040
496499161,9cyclictest10033-21apt-get07:20:123
496499160,16cyclictest0-21swapper/309:20:503
496199169,2cyclictest25564-21rm12:40:002
496199163,9cyclictest8752-21apt-get09:55:152
496199162,14cyclictest0-21swapper/211:32:082
4961991612,3cyclictest1594-21ls12:05:272
496199160,15cyclictest0-21swapper/208:20:142
495999162,1cyclictest0-21swapper/112:02:081
495999160,1cyclictest0-21swapper/109:43:351
495599165,5cyclictest0-21swapper/011:05:140
4955991615,1cyclictest0-21swapper/009:05:140
4955991615,1cyclictest0-21swapper/009:05:130
496499151,13cyclictest0-21swapper/310:43:093
496499150,14cyclictest0-21swapper/312:00:463
496499150,14cyclictest0-21swapper/309:57:083
4961991514,1cyclictest0-21swapper/208:45:152
4961991512,2cyclictest21772-21pmu-power11:00:222
495999156,5cyclictest225-21systemd-journal08:17:001
4959991513,2cyclictest0-21swapper/111:18:231
495599152,6cyclictest29467-21basename11:25:000
495599152,6cyclictest27632-21munin-run09:05:000
4955991510,4cyclictest1657-21sh09:11:400
496499147,3cyclictest0-21swapper/310:15:113
496499143,7cyclictest30608-21/usr/sbin/munin11:25:163
496499143,10cyclictest30015-21diskmemload12:35:563
496499142,11cyclictest25943-21sensors09:45:243
4964991411,2cyclictest25537-21wc07:50:193
4961991411,2cyclictest3253-21cut10:30:172
4961991411,2cyclictest25317-21wc09:00:002
496199140,1cyclictest0-21swapper/209:29:012
495999143,5cyclictest18103-21cron08:45:001
495999140,12cyclictest9105-21tr08:25:141
495999140,12cyclictest0-21swapper/112:14:281
4955991411,2cyclictest24566-21grep11:40:250
94562130,0sleep00-21swapper/012:10:200
496499139,3cyclictest17297-21grep11:55:283
4964991312,1cyclictest0-21swapper/308:00:123
4964991310,2cyclictest7658-21basename07:15:153
4964991310,2cyclictest5076-21sh11:10:143
4964991310,2cyclictest32291-21sed09:30:163
4964991310,2cyclictest28347-21grep10:45:163
4964991310,2cyclictest24565-21cat08:55:263
4964991310,2cyclictest20011-21sed08:45:273
4964991310,2cyclictest19843-21cron10:59:593
4964991310,2cyclictest1870-21seq12:05:303
4964991310,2cyclictest17231-21tr09:40:193
496499130,1cyclictest0-21swapper/312:17:373
496199139,3cyclictest24045-21grep12:00:132
496199139,1cyclictest32-21ksoftirqd/210:37:032
496199137,1cyclictest30015-21diskmemload10:20:162
4961991310,2cyclictest843-21rm09:50:182
4961991310,2cyclictest8058-21cut11:50:182
4961991310,2cyclictest8058-21cut11:50:182
4961991310,2cyclictest31375-21cron08:04:592
4961991310,2cyclictest30647-21grep11:25:162
4961991310,2cyclictest28271-21sh10:45:162
4961991310,2cyclictest25494-21cat07:50:192
4961991310,2cyclictest11429-21grep08:30:142
496199130,12cyclictest13907-21grep10:55:272
496199130,12cyclictest0-21swapper/210:09:002
495999139,3cyclictest32226-21ntpq11:45:191
495999131,2cyclictest30015-21diskmemload10:42:151
4959991312,1cyclictest0-21swapper/109:28:151
4959991312,0cyclictest0-21swapper/112:07:561
4959991310,2cyclictest8833-21wc09:35:201
4959991310,2cyclictest5938-21grep11:30:141
4959991310,2cyclictest3213-21grep08:10:261
4959991310,2cyclictest30754-21tr09:10:141
4959991310,2cyclictest22399-21tr07:45:151
4959991310,2cyclictest13165-21sed07:25:171
495999130,13cyclictest0-21swapper/110:59:381
495599139,3cyclictest26521-21rm12:20:230
495599139,3cyclictest19938-21grep10:40:160
495599139,2cyclictest7734-21grep07:15:150
495599138,3cyclictest31570-21wc08:05:000
495599138,2cyclictest27979-21rt-features10:25:220
4955991310,2cyclictest24042-21sed09:25:190
4955991310,2cyclictest21429-21chrt11:00:190
496499129,2cyclictest9802-21cat08:25:193
496499129,2cyclictest8015-21wc09:35:163
496499129,2cyclictest31490-21grep11:45:153
496499129,2cyclictest25617-21cut09:00:153
496499129,2cyclictest22716-21cut11:40:123
496499129,2cyclictest22294-21grep11:00:263
496499129,2cyclictest15662-21cron08:40:013
496499129,2cyclictest13883-21cat07:25:263
496499129,2cyclictest12835-21ssh10:35:283
496499129,2cyclictest10090-21head12:30:163
496499128,3cyclictest22747-21ls11:20:223
4964991210,1cyclictest6155-21cut11:30:153
496499120,11cyclictest25069-21grep09:25:273
496199129,2cyclictest9588-21chrt09:16:182
496199129,2cyclictest9464-21grep08:25:172
496199129,2cyclictest8988-21ntp_states09:35:222
496199129,2cyclictest5928-21awk09:15:012
496199129,2cyclictest5841-21strings11:10:202
496199129,2cyclictest28532-21sed07:55:262
496199129,2cyclictest22817-21cat09:25:002
496199129,2cyclictest12668-21grep07:25:142
496199128,3cyclictest19712-21taskset10:00:572
496199121,2cyclictest0-21swapper/209:05:132
496199121,2cyclictest0-21swapper/209:05:132
4961991210,1cyclictest6885-21grep07:10:262
496199120,1cyclictest0-21swapper/211:35:172
496199120,1cyclictest0-21swapper/211:21:092
496199120,11cyclictest16127-21grep11:55:182
495999129,2cyclictest9787-21ssh10:15:141
495999129,2cyclictest7348-21cron07:15:001
495999129,2cyclictest31843-21grep11:25:261
495999129,2cyclictest29538-21grep09:05:261
495999129,2cyclictest29538-21grep09:05:261
495999129,2cyclictest29154-21ssh11:05:141
495999129,2cyclictest26775-21wc09:00:221
495999129,2cyclictest25757-21wc09:45:211
495999129,2cyclictest25455-21grep10:05:141
495999129,2cyclictest16698-21sed09:20:241
495999127,2cyclictest19153-21taskset07:36:011
4959991212,0cyclictest0-21swapper/109:30:161
4959991210,1cyclictest126092sleep109:56:561
495999120,12cyclictest0-21swapper/111:38:371
495999120,12cyclictest0-21swapper/110:28:391
495999120,11cyclictest0-21swapper/110:02:021
495599129,2cyclictest8793-21grep12:30:000
495599129,2cyclictest7664-21sh12:10:000
495599129,2cyclictest7360-21sed09:15:170
495599129,2cyclictest7359-21sh07:15:000
495599129,2cyclictest571-21tr09:50:160
495599129,2cyclictest3376-21ssh10:50:110
495599129,2cyclictest19352-21grep10:20:200
495599129,2cyclictest14511-21sh07:30:000
495599129,2cyclictest14299-21grep08:35:170
495599128,3cyclictest19844-21cron11:00:000
495599128,3cyclictest17259-21ssh12:15:150
495599127,3cyclictest6975-21apache_processe11:50:120
495599127,3cyclictest6975-21apache_processe11:50:110
495599120,2cyclictest0-21swapper/011:25:060
496499119,1cyclictest28626-21chrt07:55:293
496499118,2cyclictest8030-21ssh12:10:073
496499118,2cyclictest7133-21ssh09:15:163
496499118,2cyclictest7040-21grep08:20:173
496499118,2cyclictest6315-21ntpq07:10:203
496499118,2cyclictest4649-21ssh09:52:193
496499118,2cyclictest3681-21ssh10:10:423
496499118,2cyclictest3681-21ssh10:10:413
496499118,2cyclictest3510-21tr10:50:143
496499118,2cyclictest30664-21cut09:10:143
496499118,2cyclictest30563-21ssh10:07:543
496499118,2cyclictest29371-21ssh12:22:223
496499118,2cyclictest29276-21taskset09:05:223
496499118,2cyclictest29276-21taskset09:05:223
496499118,2cyclictest2353-21head08:10:173
496499118,2cyclictest22310-21cat08:50:243
496499118,2cyclictest18056-21ssh10:20:103
496499118,2cyclictest17931-21ssh10:00:163
496499118,2cyclictest1662-21chrt11:08:083
496499118,2cyclictest15280-21tr07:30:143
496499118,2cyclictest13957-21rm11:54:173
496499118,2cyclictest13957-21rm11:54:173
496499118,2cyclictest13731-21expr11:15:143
496499118,2cyclictest11417-21grep08:30:143
496499117,3cyclictest18146-21head07:35:163
496199119,1cyclictest24419-21tr12:20:002
496199118,2cyclictest684-21ssh11:07:222
496199118,2cyclictest4705-21head08:15:172
496199118,2cyclictest4187-21head10:50:162
496199118,2cyclictest356-21sed11:45:252
496199118,2cyclictest32362-21expr08:05:202
496199118,2cyclictest31623-21grep09:30:132
496199118,2cyclictest3084-21sort10:10:232
496199118,2cyclictest3084-21sort10:10:232
496199118,2cyclictest27143-21grep10:25:172
496199118,2cyclictest26754-21seq09:45:452
496199118,2cyclictest2107-21ssh12:25:202
496199118,2cyclictest20435-21grep07:40:172
496199118,2cyclictest2026-21cat08:10:142
496199118,2cyclictest16943-21wc08:40:182
496199118,2cyclictest16577-21ssh11:16:282
496199118,2cyclictest16082-21ssh09:40:132
496199118,2cyclictest14278-21head08:35:182
496199118,2cyclictest11259-21grep07:20:212
496199110,6cyclictest0-21swapper/210:15:142
496199110,10cyclictest14446-21chrt12:32:532
495999119,2cyclictest0-21swapper/108:55:141
495999119,1cyclictest2787-21sed12:25:281
495999118,2cyclictest4318-21ssh10:11:121
495999118,2cyclictest4318-21ssh10:11:121
495999118,2cyclictest32474-21idleruntime08:05:201
495999118,2cyclictest22670-21ssh12:37:461
495999118,2cyclictest20042-21grep08:45:271
495999118,2cyclictest15933-21cut07:30:191
495999118,2cyclictest12924-21ssh09:18:551
495999118,2cyclictest12879-21grep08:30:261
495999118,2cyclictest11803-21chrt07:20:491
495999118,2cyclictest10029-21cut12:30:161
495999117,3cyclictest27481-21sort07:55:171
495999114,3cyclictest8560-21turbostat.cron08:25:001
495999110,10cyclictest9781-21awk07:20:011
495599119,1cyclictest25477-21sh10:05:130
495599118,2cyclictest9570-21sed09:55:190
495599118,2cyclictest9530-21ssh10:15:040
495599118,2cyclictest767-21ssh11:45:360
495599118,2cyclictest5937-21wget11:30:140
495599118,2cyclictest25296-21wc12:00:210
495599118,2cyclictest24531-21chrt07:50:130
495599118,2cyclictest23448-21ssh10:03:520
495599118,2cyclictest20810-21grep08:50:130
495599118,2cyclictest20550-21ssh11:19:340
495599118,2cyclictest19368-21ssh09:22:250
495599118,2cyclictest18549-21ssh09:40:530
495599118,2cyclictest16202-21tr11:55:170
495599118,2cyclictest12308-21ssh12:31:120
495599118,2cyclictest12034-21cat10:35:200
495599118,2cyclictest10990-21wc07:20:180
495599117,2cyclictest8852-21cut08:25:150
495599113,5cyclictest15517-21grep11:35:190
495599111,2cyclictest225-21systemd-journal08:35:000
4955991110,1cyclictest0-21swapper/009:37:560
495599110,10cyclictest0-21swapper/012:35:100
496499108,1cyclictest2571-21wget10:30:153
496499108,1cyclictest21147-21chrt07:40:243
496499108,1cyclictest1232-21chrt08:08:523
496499100,9cyclictest16863-21cat08:40:183
496199108,1cyclictest9117-21cat07:15:272
496199108,1cyclictest27456-21taskset09:03:012
496199108,1cyclictest22565-21expr07:45:152
496199108,1cyclictest21953-21cut08:50:202
496199108,1cyclictest14836-21cat07:30:012
496199107,2cyclictest19509-21basename07:40:002
495999108,1cyclictest29232-21taskset08:00:031
495999108,1cyclictest21308-21sed08:50:151
495999108,1cyclictest13646-21basename08:35:151
495999107,2cyclictest25220-21grep07:50:161
495999107,2cyclictest19989-21tr07:40:141
495599108,1cyclictest8279-21chrt08:21:390
495599108,1cyclictest4065-21cut08:15:140
495599108,1cyclictest22133-21chrt07:45:130
495599108,1cyclictest18565-21cut07:35:210
495599108,1cyclictest17877-21chrt08:42:420
495599108,1cyclictest16963-21taskset07:33:230
495599108,1cyclictest10752sleep008:06:520
495599107,2cyclictest27186-21cat07:55:150
495599107,2cyclictest20131-21taskset08:45:400
495599107,2cyclictest20107-21cat07:40:140
495599106,2cyclictest2220-21cut08:10:170
49649997,1cyclictest23441-21gzip07:45:223
49559997,1cyclictest236992sleep008:55:160
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional