You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-29 - 19:31

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot7s.osadl.org (updated Sat Aug 29, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
14119210478,9sleep20-21swapper/207:09:192
1416929683,9sleep00-21swapper/007:09:570
1390828672,10sleep30-21swapper/307:06:343
1383928467,13sleep10-21swapper/107:05:401
228062510,0sleep10-21swapper/109:19:141
14263992422,1cyclictest0-21swapper/111:14:121
278732230,1sleep3381rcuc/308:45:143
1426999220,21cyclictest0-21swapper/312:23:473
1426799220,22cyclictest0-21swapper/212:23:472
1426399221,21cyclictest7010-21diskmemload11:18:081
14261992221,1cyclictest0-21swapper/009:45:430
1426199220,0cyclictest0-21swapper/011:43:490
1426399210,21cyclictest0-21swapper/111:27:561
14267991917,1cyclictest25816-21awk10:00:002
1426799190,14cyclictest0-21swapper/210:25:142
1426399190,19cyclictest0-21swapper/112:01:541
1426399190,17cyclictest0-21swapper/111:30:311
14263991817,1cyclictest7010-21diskmemload10:36:091
313602170,0sleep20-21swapper/211:39:452
1426999171,2cyclictest0-21swapper/312:16:353
1426999171,2cyclictest0-21swapper/312:16:343
14269991711,4cyclictest24629-21sh10:56:243
1426999170,17cyclictest0-21swapper/309:35:223
1426399177,6cyclictest14253-21cyclictest10:23:061
1426399171,8cyclictest537-21snmpd10:44:531
1426199170,1cyclictest0-21swapper/007:10:130
1426999160,2cyclictest25866-21grep11:55:173
1426799160,16cyclictest0-21swapper/210:40:242
1426399164,11cyclictest11359-21tune2fs10:10:161
1426399160,2cyclictest7010-21diskmemload11:43:111
1426399160,16cyclictest0-21swapper/110:49:351
1426199160,16cyclictest0-21swapper/010:43:540
1426199160,14cyclictest0-21swapper/011:15:150
1426999154,3cyclictest225-21systemd-journal09:35:013
14269991514,1cyclictest0-21swapper/310:50:193
14269991511,3cyclictest39-21ksoftirqd/310:05:193
1426799152,11cyclictest0-21swapper/211:40:142
1426799150,14cyclictest0-21swapper/210:13:482
14263991510,2cyclictest26865-21cron12:35:001
14261991511,2cyclictest0-21swapper/011:24:370
1426199150,0cyclictest0-21swapper/012:16:050
1426199150,0cyclictest0-21swapper/012:16:040
308042140,1sleep00-21swapper/010:21:300
1426999149,2cyclictest0-21swapper/312:14:293
14269991410,3cyclictest26911-21ls10:20:003
1426999140,12cyclictest0-21swapper/309:54:583
1426799145,6cyclictest32489-21ssh09:44:352
14267991410,3cyclictest29574-21chrt09:23:012
14267991410,3cyclictest2098-21/usr/sbin/munin09:45:182
14267991410,3cyclictest10375-21wc09:30:252
1426799140,12cyclictest0-21swapper/210:07:382
14263991410,4cyclictest38150irq/131-snd_hda11:46:021
1426399140,2cyclictest0-21swapper/111:51:131
1426399140,1cyclictest7010-21diskmemload11:35:401
1426399140,1cyclictest7010-21diskmemload09:42:251
14261991413,1cyclictest0-21swapper/011:53:470
1426199140,13cyclictest0-21swapper/011:46:450
1426199140,0cyclictest0-21swapper/012:11:280
1426999133,7cyclictest10458-21sshd09:11:073
1426999132,10cyclictest9609-21rm11:06:453
14269991311,1cyclictest0-21swapper/308:55:133
14269991310,2cyclictest30834-21tune2fs08:50:183
14269991310,2cyclictest12037-21sort10:30:143
14269991310,2cyclictest12037-21sort10:30:133
1426999130,1cyclictest0-21swapper/309:27:483
1426999130,12cyclictest639-21sleep09:45:003
1426799139,3cyclictest31487-21find11:00:222
1426799139,3cyclictest28598-21ls12:35:222
1426799139,3cyclictest26717-21grep07:35:152
1426799139,2cyclictest19796-21cut12:30:172
1426799135,4cyclictest12401-21/usr/sbin/munin10:30:152
1426799135,4cyclictest12401-21/usr/sbin/munin10:30:142
14267991311,1cyclictest0-21swapper/208:50:152
14267991310,2cyclictest6516-21sed09:05:272
14267991310,2cyclictest6516-21sed09:05:262
14267991310,2cyclictest30140-21tr07:40:222
14267991310,2cyclictest21701-21tr10:35:232
14267991310,2cyclictest199642sleep212:10:342
14267991310,2cyclictest13964-21grep10:50:182
1426799130,12cyclictest0-21swapper/211:10:142
14263991312,1cyclictest0-21swapper/112:29:561
14263991310,2cyclictest6555-21ssh12:22:131
1426399130,13cyclictest0-21swapper/112:06:461
1426199139,3cyclictest27876-21ssh12:35:170
1426199139,3cyclictest18500-21mailstats09:35:240
1426199139,2cyclictest13-21ksoftirqd/011:26:290
14261991313,0cyclictest0-21swapper/009:55:060
14261991310,2cyclictest4135-21grep09:00:260
14261991310,2cyclictest24687-21sed08:35:270
14261991310,2cyclictest20089-21sort07:20:200
14261991310,2cyclictest19507-21cat08:25:260
1426199130,1cyclictest0-21swapper/009:24:000
1426199130,12cyclictest20873-21chrt11:33:060
1426999129,2cyclictest804-21grep11:40:173
1426999129,2cyclictest3025-21sed07:50:263
1426999129,2cyclictest2800-21munin-plugin-st12:40:013
1426999129,2cyclictest21430-21grep08:30:173
1426999129,2cyclictest11653-21ssh10:49:173
1426999129,2cyclictest11507-21sort08:10:173
1426999129,2cyclictest1047-21cron07:50:003
1426999128,3cyclictest13206-21ssh12:25:503
1426999121,6cyclictest0-21swapper/310:10:013
1426999121,10cyclictest14585-21ssh11:48:443
1426999120,6cyclictest7010-21diskmemload11:51:133
1426799129,2cyclictest9665-21tr12:05:152
1426799129,2cyclictest656-21grep08:55:162
1426799129,2cyclictest4258-21grep07:55:152
1426799129,2cyclictest4137-21sed09:00:272
1426799129,2cyclictest31392-21grep11:20:142
1426799129,2cyclictest28249-21sed12:15:272
1426799129,2cyclictest28249-21sed12:15:262
1426799129,2cyclictest20325-21cat08:30:012
1426799129,2cyclictest19694-21ssh10:15:162
1426799129,2cyclictest1756-21cut12:00:152
1426799129,2cyclictest12601-21head08:10:262
1426799129,2cyclictest11062-21tr12:25:182
1426799128,3cyclictest10299-21grep11:45:262
1426799127,4cyclictest32-21ksoftirqd/211:50:112
1426799125,7cyclictest9734-21apache210:21:042
1426799124,2cyclictest225-21systemd-journal07:45:032
1426799122,10cyclictest0-21swapper/209:52:562
14267991210,1cyclictest7010-21diskmemload11:34:202
14267991210,1cyclictest27169-21grep10:00:172
1426399129,2cyclictest9868-21tr09:50:151
1426399129,2cyclictest3746-21cut07:55:141
1426399129,2cyclictest2616-21tr10:05:161
1426399129,2cyclictest2244-21perf09:00:011
1426399129,2cyclictest18073-21cut12:10:131
1426399129,2cyclictest15417-21grep07:10:181
1426399128,3cyclictest18976-21cron07:20:011
1426399127,2cyclictest449-21irqbalance12:15:121
1426399127,2cyclictest449-21irqbalance12:15:111
1426399126,5cyclictest28066-21tr08:45:151
1426399123,5cyclictest7010-21diskmemload09:34:011
14263991210,1cyclictest2654-21tr09:00:151
14263991210,1cyclictest14859-21wc08:15:261
1426399120,11cyclictest19915-21sed10:15:181
1426199129,2cyclictest3494-21cut12:20:190
1426199129,2cyclictest30792-21grep08:50:170
1426199129,2cyclictest30262-21sh11:57:560
1426199129,2cyclictest2957-21grep07:50:260
1426199129,2cyclictest19921-21grep10:15:180
1426199129,2cyclictest19358-21grep12:30:150
1426199129,2cyclictest1248-21grep12:00:130
1426199129,2cyclictest10417-21tail09:50:170
1426199128,3cyclictest1192-21ssh10:04:360
1426199120,6cyclictest26154-21basename07:35:010
1426999119,2cyclictest30470-21sh11:39:033
1426999119,1cyclictest5061-21cut07:55:223
1426999119,1cyclictest31948-21sed11:20:163
1426999119,1cyclictest3083-21sed09:45:273
1426999119,1cyclictest26884-21grep07:35:153
1426999119,1cyclictest24760-21head07:30:163
1426999119,1cyclictest21901-21grep07:25:163
1426999119,1cyclictest0-21swapper/311:12:353
1426999118,2cyclictest27831-21ssh10:00:233
1426999118,2cyclictest27551-21hostname10:20:153
1426999118,2cyclictest25654-21expr08:40:163
1426999118,2cyclictest20661-21rm11:32:563
1426999118,2cyclictest15802-21ssh09:15:063
1426999118,2cyclictest1446-21expr12:00:143
1426999118,2cyclictest13998-21grep08:15:173
1426999118,2cyclictest13207-21sh11:28:393
1426999112,4cyclictest225-21systemd-journal12:10:013
1426999111,5cyclictest5422-21munin-run11:05:013
1426999111,2cyclictest0-21swapper/311:17:453
1426799118,2cyclictest6736-21tail11:05:172
1426799118,2cyclictest5991-21tr10:45:212
1426799118,2cyclictest5012-21ssh09:27:382
1426799118,2cyclictest25208-21cut11:55:152
1426799118,2cyclictest22327-21sort07:25:192
1426799118,2cyclictest1958-21cut07:50:172
1426799118,2cyclictest17141-21wc09:15:202
1426799118,2cyclictest15410-21munin-run08:20:002
1426799118,2cyclictest14650-21ssh11:29:452
1426799117,2cyclictest21703-21tr10:55:142
1426799117,2cyclictest19993-21grep07:20:182
1426799116,4cyclictest55650irq/127-enp2s0-09:10:152
1426799110,10cyclictest26642-21ssh11:16:572
1426799110,10cyclictest0-21swapper/209:35:252
1426399119,1cyclictest9227-21grep08:05:181
1426399119,1cyclictest27239-21cut07:35:161
1426399119,1cyclictest26191-21grep09:20:261
1426399119,1cyclictest24252-21tr08:35:221
1426399119,1cyclictest21878-21grep08:30:221
1426399119,1cyclictest17757-21tail09:35:191
1426399119,1cyclictest11655-21sed08:10:161
1426399118,2cyclictest8687-21chrt11:06:021
1426399118,2cyclictest7302-21chrt10:26:541
1426399118,2cyclictest4368-21ssh09:27:081
1426399118,2cyclictest30362-21rm11:00:151
1426399118,2cyclictest29334-21tail12:35:281
1426399118,2cyclictest28360-21ssh11:56:271
1426399118,2cyclictest27519-21wc10:00:211
1426399118,2cyclictest21718-21grep07:25:141
1426399118,2cyclictest20417-21ntpq07:20:221
1426399118,2cyclictest12116-21tr10:30:151
1426399118,2cyclictest12116-21tr10:30:151
1426399117,3cyclictest24071-21sh10:55:591
1426399116,4cyclictest13206-21cut10:50:151
1426399116,2cyclictest13142-21sh09:13:141
1426399114,6cyclictest449-21irqbalance09:56:321
1426399112,8cyclictest1643-21ssh11:21:111
1426399110,10cyclictest19996-21chrt08:27:571
1426199119,1cyclictest13538-21grep08:15:140
1426199118,2cyclictest9755-21grep08:05:230
1426199118,2cyclictest9734-21apache209:11:200
1426199118,2cyclictest9109-21ssh11:06:220
1426199118,2cyclictest30424-21cat07:40:270
1426199118,2cyclictest29092-21chrt09:41:530
1426199118,2cyclictest27922-21basename08:45:150
1426199118,2cyclictest26614-21ls08:40:230
1426199118,2cyclictest17931-21cat08:25:000
1426199118,2cyclictest16514-21ssh09:15:160
1426199118,2cyclictest15576-21ssh10:50:500
1426199118,2cyclictest14687-21grep11:10:150
1426199118,2cyclictest12742-21sed12:25:300
1426199118,2cyclictest12660-21ssh10:10:260
1426199117,1cyclictest5947-21ssh10:25:510
1426199110,2cyclictest0-21swapper/009:26:170
1426199110,10cyclictest7591-21ssh10:46:090
1426199110,10cyclictest23611-21ssh11:35:020
1426999108,2cyclictest9058-21dnf08:05:163
1426999108,2cyclictest24947-21taskset08:37:453
1426999108,2cyclictest19578-21sh09:55:263
1426999108,1cyclictest81832sleep308:04:523
1426999108,1cyclictest5198-21cat09:05:163
1426999108,1cyclictest5198-21cat09:05:153
1426999108,1cyclictest3269-21ssh10:25:063
1426999108,1cyclictest2878-21expr09:00:163
1426999108,1cyclictest28451-21ssh10:40:013
1426999108,1cyclictest18787-21ssh12:30:023
1426999108,1cyclictest14877-21cat07:10:143
1426999107,2cyclictest308732sleep307:42:033
1426999107,2cyclictest24212-21sh09:20:093
1426999107,2cyclictest20941-21sed07:20:273
1426999107,2cyclictest17300-21grep08:20:253
1426999104,3cyclictest225-21systemd-journal08:27:493
1426999103,6cyclictest21377-21if_enp2s010:35:213
1426799109,0cyclictest0-21swapper/207:30:132
1426799108,1cyclictest23549-21sed08:35:162
1426799108,1cyclictest187532sleep207:17:112
1426799108,1cyclictest165352sleep207:14:182
1426799108,1cyclictest10713-21sort08:10:002
1426799107,2cyclictest17388-21grep08:20:262
1426399108,1cyclictest6100-21chrt08:00:131
1426399108,1cyclictest31107-21cron07:45:011
1426399107,2cyclictest5826-21uname09:05:201
1426399107,2cyclictest5826-21uname09:05:191
1426399107,2cyclictest32019-21taskset08:51:511
1426399107,2cyclictest31486-21cut07:45:131
1426399107,2cyclictest1862-21ssh09:45:171
1426399106,3cyclictest24060-21cut07:30:131
1426199108,1cyclictest8384-21ssh09:30:080
1426199108,1cyclictest7436-21run-parts08:00:230
1426199108,1cyclictest4186-21expr07:55:160
1426199108,1cyclictest29816-21ssh11:00:060
1426199108,1cyclictest285132sleep007:37:280
1426199108,1cyclictest21981-21expr07:25:150
1426199108,1cyclictest21699-21ntpq10:35:230
1426199108,1cyclictest2152-21sh10:05:090
1426199108,1cyclictest21249-21ssh10:55:080
1426199108,1cyclictest17747-21if_enp2s007:15:180
1426199108,1cyclictest129082sleep008:13:460
1426199107,2cyclictest9080-21ssh12:05:010
1426199107,2cyclictest32074-21grep07:45:160
1426199107,2cyclictest20995-21expr08:30:150
1426199107,2cyclictest1327-21cut08:55:220
1426199107,2cyclictest13039-21cat10:30:200
1426199107,2cyclictest13039-21cat10:30:190
1426199100,9cyclictest5314-21expr09:05:160
1426199100,9cyclictest5314-21expr09:05:150
142699998,1cyclictest17978-21uname07:15:343
142679997,1cyclictest7873-21chrt08:00:512
142679997,1cyclictest27357-21chrt08:42:582
142679996,2cyclictest29806-21taskset08:48:562
142679996,2cyclictest21416-21grep08:30:182
142639997,2cyclictest15690-21apt-get08:20:121
142639996,2cyclictest25571-21wc08:40:161
142639996,2cyclictest1583-21tr07:50:151
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional