You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-04 - 18:39

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot7s.osadl.org (updated Mon May 04, 2026 12:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1264429787,6sleep20-21swapper/207:09:332
1267229280,8sleep00-21swapper/007:09:560
1236428666,16sleep10-21swapper/107:05:551
1255728572,9sleep30-21swapper/307:08:253
94322440,0sleep00-21swapper/010:48:210
1276899300,29cyclictest0-21swapper/112:10:171
12768992625,1cyclictest0-21swapper/112:09:281
1276499260,17cyclictest0-21swapper/011:14:400
12764992423,1cyclictest0-21swapper/010:14:300
12773992321,1cyclictest39-21ksoftirqd/307:20:013
12768992319,4cyclictest25-21ksoftirqd/111:52:541
12773992219,3cyclictest39-21ksoftirqd/310:50:153
1276899220,5cyclictest0-21swapper/110:19:001
1277399210,2cyclictest0-21swapper/311:01:593
1276899210,20cyclictest0-21swapper/109:32:061
1276899210,18cyclictest5779-21diskmemload10:23:421
1276899210,0cyclictest0-21swapper/109:58:241
1276899200,19cyclictest30322-21ssh11:39:401
12773991917,1cyclictest39-21ksoftirqd/309:58:413
1276899193,15cyclictest5779-21diskmemload10:00:261
1276899190,3cyclictest0-21swapper/110:08:271
108162190,0sleep20-21swapper/211:27:262
1277399182,15cyclictest0-21swapper/309:45:143
12773991815,3cyclictest21705-21kworker/u8:011:30:263
12773991815,1cyclictest141rcu_preempt11:39:453
12773991814,1cyclictest39-21ksoftirqd/310:10:183
12773991813,2cyclictest0-21swapper/310:30:433
1277399180,1cyclictest0-21swapper/309:35:223
12771991817,1cyclictest32-21ksoftirqd/209:25:422
1277199180,1cyclictest0-21swapper/211:15:142
1277199180,1cyclictest0-21swapper/211:15:132
1276899186,8cyclictest20938-21kworker/1:011:00:181
12768991818,0cyclictest0-21swapper/110:50:191
1276899180,17cyclictest0-21swapper/111:16:231
1276899180,17cyclictest0-21swapper/111:16:221
1277399175,8cyclictest39-21ksoftirqd/312:16:013
12773991717,0cyclictest0-21swapper/311:14:503
12773991714,2cyclictest39-21ksoftirqd/311:05:233
1277199175,7cyclictest18272-21fschecks_count12:30:172
12768991715,1cyclictest0-21swapper/111:43:021
1276899170,1cyclictest0-21swapper/110:28:501
1276899170,16cyclictest0-21swapper/109:18:561
1276499175,3cyclictest537-21snmpd09:12:030
1276499171,3cyclictest141rcu_preempt11:56:180
12764991713,3cyclictest0-21swapper/010:35:130
1276499170,15cyclictest0-21swapper/012:29:350
1277399166,5cyclictest39-21ksoftirqd/312:25:593
1277399164,5cyclictest21829-21runrttasks08:35:143
1277399161,0cyclictest0-21swapper/312:13:303
12771991615,1cyclictest5779-21diskmemload10:38:122
12771991613,2cyclictest2408-21/usr/sbin/munin12:20:222
12768991614,2cyclictest0-21swapper/112:20:551
12768991612,2cyclictest0-21swapper/109:24:211
1276899160,13cyclictest0-21swapper/109:45:481
1276499163,6cyclictest22440-21munin-run11:35:010
1276499162,1cyclictest0-21swapper/007:20:130
1276499160,3cyclictest0-21swapper/012:31:490
201742150,1sleep10-21swapper/109:37:301
1277399155,3cyclictest0-21swapper/309:30:203
12773991512,2cyclictest39-21ksoftirqd/308:10:233
12773991511,3cyclictest39-21ksoftirqd/311:52:323
1277399151,12cyclictest0-21swapper/309:10:143
12773991511,1cyclictest39-21ksoftirqd/311:47:553
1277199159,5cyclictest12860-21tail10:50:172
1277199156,7cyclictest16084-21ssh12:10:002
1277199156,4cyclictest225-21systemd-journal12:18:202
12768991514,1cyclictest0-21swapper/111:10:191
12768991513,1cyclictest0-21swapper/111:59:361
12768991513,1cyclictest0-21swapper/111:33:481
12768991513,1cyclictest0-21swapper/108:45:011
12768991513,1cyclictest0-21swapper/108:45:001
1276899150,2cyclictest5779-21diskmemload10:13:121
1276899150,1cyclictest0-21swapper/111:05:151
1276899150,15cyclictest0-21swapper/112:18:011
1276499154,4cyclictest0-21swapper/012:00:130
12764991512,2cyclictest1235-21ntpd09:55:460
1276499150,7cyclictest31676-21grep09:25:150
1277399141,12cyclictest0-21swapper/309:53:013
12773991410,3cyclictest15310-21grep09:15:163
1277399140,3cyclictest141rcu_preempt11:20:093
1277199141,1cyclictest0-21swapper/212:36:122
12771991411,2cyclictest32115-21sed11:20:262
1277199140,14cyclictest0-21swapper/211:46:282
1276899144,6cyclictest225-21systemd-journal08:17:001
1276899142,6cyclictest0-21swapper/112:25:141
1276899142,11cyclictest0-21swapper/108:35:151
1276899141,2cyclictest31247-21rm09:44:261
12768991412,1cyclictest0-21swapper/108:55:121
12768991411,2cyclictest5758-21cron09:10:011
12768991410,3cyclictest18959-21ls12:30:221
1276899140,3cyclictest0-21swapper/110:31:061
1276499146,8cyclictest0-21swapper/009:20:140
1276499142,10cyclictest0-21swapper/011:25:120
12764991411,2cyclictest8210-21cron09:50:000
12764991410,3cyclictest10657-21sed09:50:270
1276499140,1cyclictest0-21swapper/012:05:170
1277399139,3cyclictest24385-21grep09:40:143
1277399134,8cyclictest7794-21/usr/sbin/munin11:25:223
1277399132,11cyclictest29187-21kworker/u8:111:15:093
1277399132,11cyclictest29187-21kworker/u8:111:15:093
12773991310,2cyclictest39-21ksoftirqd/312:20:143
12773991310,2cyclictest39-21ksoftirqd/307:40:003
12773991310,2cyclictest32333-21grep08:55:193
12773991310,2cyclictest26958-21grep12:35:173
1277399130,2cyclictest0-21swapper/312:04:173
1277399130,1cyclictest141rcu_preempt10:15:183
1277399130,1cyclictest0-21swapper/312:32:403
1277399130,12cyclictest31932-21sed09:25:143
1277399130,12cyclictest2970-21grep07:55:163
1277399130,11cyclictest0-21swapper/311:57:353
1277399130,10cyclictest0-21swapper/310:00:143
1277199136,7cyclictest0-21swapper/212:11:022
1277199133,9cyclictest0-21swapper/208:07:362
12771991311,1cyclictest15374-21apache_accesses07:15:122
12771991311,1cyclictest10224-21uniq08:10:172
12771991310,2cyclictest9413-21grep09:50:182
12771991310,2cyclictest30247-21tr11:00:202
12771991310,2cyclictest25260-21head11:55:222
12771991310,2cyclictest22277-21wc10:55:262
12771991310,2cyclictest17550-21grep09:35:272
12771991310,2cyclictest15386-21cat08:20:202
12771991310,2cyclictest14140-21ntpq07:10:212
1277199130,1cyclictest0-21swapper/209:45:482
1277199130,1cyclictest0-21swapper/209:30:132
12768991312,1cyclictest0-21swapper/111:28:381
12768991312,1cyclictest0-21swapper/110:45:151
12768991311,2cyclictest0-21swapper/109:53:511
12768991310,2cyclictest26391-21grep08:45:161
12768991310,2cyclictest21268-21grep08:35:001
12768991310,2cyclictest14991-21cron11:50:011
12768991310,2cyclictest14436-21cut08:20:151
1276899130,1cyclictest0-21swapper/112:38:501
1276499139,3cyclictest15840-21tr11:50:150
1276499139,3cyclictest14993-21cron11:50:010
1276499133,6cyclictest537-21snmpd10:08:050
1276499132,8cyclictest18909-21turbostat.cron10:35:000
12764991311,2cyclictest13-21ksoftirqd/009:32:060
12764991310,2cyclictest6767-21cron11:45:000
12764991310,2cyclictest3726-21grep09:05:140
12764991310,2cyclictest29653-21sed10:40:270
12764991310,2cyclictest26424-21grep12:35:150
12764991310,2cyclictest22403-21wc10:55:260
123112130,0chrt0-21swapper/312:07:103
1277399129,2cyclictest3426-21cut10:25:173
1277399129,2cyclictest32553-21grep11:40:193
1277399129,2cyclictest29647-21cut08:50:153
1277399129,2cyclictest29623-21perf07:45:003
1277399129,2cyclictest23689-21sort09:20:173
1277399129,2cyclictest2005-21munin-run07:55:003
1277399129,2cyclictest13741-21grep07:10:173
12773991211,1cyclictest39-21ksoftirqd/310:09:203
1277399120,12cyclictest0-21swapper/310:36:363
1277199129,2cyclictest792-21ssh10:05:002
1277199129,2cyclictest28463-21uniq10:40:172
1277199129,2cyclictest22290-21tail08:35:172
1277199129,2cyclictest20244-21grep08:30:192
1277199129,2cyclictest16590-21sed11:30:272
1277199129,2cyclictest12969-21grep10:30:272
1277199128,3cyclictest25014-21grep08:40:212
1277199128,3cyclictest25014-21grep08:40:202
1277199128,3cyclictest23581-21cat07:30:202
12771991210,1cyclictest10057-21grep12:25:182
1277199120,11cyclictest32169-21cat07:50:002
1277199120,11cyclictest0-21swapper/209:58:092
1276899129,2cyclictest32531-21ssh09:25:191
1276899129,2cyclictest31159-21head11:20:171
1276899129,2cyclictest15997-21cut07:15:171
1276899128,3cyclictest31504-21sed07:45:271
1276899126,5cyclictest2935-21cut07:55:161
1276899125,6cyclictest24782-21wc07:35:001
1276899122,9cyclictest20443-21ssh10:35:181
1276899122,7cyclictest7589-21ls12:05:001
12768991210,2cyclictest0-21swapper/110:55:341
1276499129,2cyclictest3871-21grep07:55:260
1276499129,2cyclictest3224-21ssh10:25:170
1276499129,2cyclictest31506-21tr08:55:140
1276499129,2cyclictest29858-21wc08:50:160
1276499129,2cyclictest29049-21aten2_r7power_e11:00:140
1276499129,2cyclictest28509-21cat10:20:270
1276499129,2cyclictest27786-21grep07:40:140
1276499129,2cyclictest24579-21smart_sda11:35:260
1276499129,2cyclictest2400-21tr09:00:200
1276499129,2cyclictest20855-21grep08:30:250
1276499129,2cyclictest1728-21sed12:20:160
1276499129,2cyclictest14624-21apache_processe09:15:130
1276499129,2cyclictest14125-21ls08:19:590
1276499128,3cyclictest24249-21grep09:40:140
1276499128,3cyclictest23125-21grep08:35:260
1276499128,3cyclictest22007-21tr11:15:160
1276499128,3cyclictest22007-21tr11:15:150
1276499128,3cyclictest16154-21sed07:15:170
1276499121,4cyclictest30120-21taskset10:02:360
1277399119,1cyclictest17860-21grep08:25:183
1277399118,2cyclictest4595-21grep10:45:173
1277399118,2cyclictest3774-21cut09:05:153
1277399118,2cyclictest27560-21sort10:20:173
1277399118,2cyclictest2267-21ssh10:44:153
1277399118,2cyclictest20595-21ssh10:55:123
1277399118,2cyclictest20000-21mii-tool08:30:183
1277399118,2cyclictest18424-21grep07:20:153
1277399118,2cyclictest13124-21tr08:15:203
1277399115,2cyclictest225-21systemd-journal08:07:213
1277199118,2cyclictest5232-21gzip10:45:222
1277199118,2cyclictest5098-21grep08:00:142
1277199118,2cyclictest4478-21ssh12:02:332
1277199118,2cyclictest3428-21head10:25:172
1277199118,2cyclictest31019-21sh11:40:012
1277199118,2cyclictest26077-21rm09:21:172
1277199118,2cyclictest25181-21tr07:35:132
1277199118,2cyclictest20747-21grep07:25:162
1277199118,2cyclictest20038-21cat10:15:262
1277199118,2cyclictest1999-21apt07:55:002
1277199118,2cyclictest18490-21grep08:25:252
1277199118,2cyclictest16615-21sed09:15:262
1277199118,2cyclictest15296-21ssh11:50:032
1277199118,2cyclictest1462-21cut09:00:132
1277199118,2cyclictest13057-21cat08:15:192
1277199118,2cyclictest11870-21ssh10:10:272
1277199115,6cyclictest0-21swapper/208:52:392
1277199115,2cyclictest5779-21diskmemload10:05:162
1277199112,2cyclictest225-21systemd-journal09:43:102
1277199110,4cyclictest0-21swapper/209:11:302
1277199110,0cyclictest0-21swapper/207:20:122
1276899118,2cyclictest6022-21ssh09:10:021
1276899118,2cyclictest5584-21grep08:00:171
1276899118,2cyclictest29696-21cat08:50:151
1276899118,2cyclictest29633-21awk07:45:001
1276899118,2cyclictest27208-21cron07:39:591
1276899118,2cyclictest20894-21grep07:25:171
1276899118,2cyclictest19875-21cron07:25:001
1276899118,2cyclictest13717-21grep07:10:161
1276899118,2cyclictest10837-21grep08:10:221
1276899117,3cyclictest28577-21grep10:40:181
1276899111,8cyclictest0-21swapper/108:25:121
12768991110,1cyclictest0-21swapper/108:05:141
1276499119,2cyclictest0-21swapper/012:10:480
1276499119,1cyclictest25376-21tr07:35:140
1276499118,2cyclictest8283-21sort08:05:210
1276499118,2cyclictest6886-21chrt08:03:560
1276499118,2cyclictest532-21grep07:50:160
1276499118,2cyclictest24708-21ssh12:15:030
1276499118,2cyclictest24538-21cut08:40:180
1276499118,2cyclictest24538-21cut08:40:180
1276499118,2cyclictest23614-21sh10:18:030
1276499118,2cyclictest22480-21chrt07:30:020
1276499118,2cyclictest22246-21cron07:29:590
1276499118,2cyclictest12890-21tail10:50:160
1276499118,2cyclictest11343-21head08:10:270
1276499112,8cyclictest31605-21ssh11:20:210
1276499110,10cyclictest15122-21basename07:15:000
1277399108,2cyclictest26744-21expr08:45:163
1277399108,1cyclictest5292-21expr08:00:143
1277399108,1cyclictest33512sleep309:04:403
1277399108,1cyclictest31773-21chrt07:47:233
1277399108,1cyclictest24786-21cat08:40:193
1277399108,1cyclictest24786-21cat08:40:183
1277399107,2cyclictest39-21ksoftirqd/307:30:193
1277399100,9cyclictest20772-21fschecks_count07:25:163
1277199108,2cyclictest16681-21rm11:11:252
1277199108,1cyclictest31644-21expr08:55:142
1277199108,1cyclictest266742sleep210:20:132
1277199108,1cyclictest2482-21sort07:55:142
1277199107,2cyclictest28128-21grep07:40:162
1277199107,2cyclictest26544-21rm11:36:412
1277199106,2cyclictest1370-21apache211:05:392
1277199100,9cyclictest0-21swapper/208:45:162
1276899108,1cyclictest17422sleep107:51:501
1276899107,2cyclictest1342-21aten2_r7power_v09:00:141
1276499108,1cyclictest27039-21expr08:45:190
1276499107,2cyclictest4840-21ssh11:05:120
1276499107,2cyclictest17550-21sed08:25:170
1276499107,2cyclictest15701-21ssh09:35:140
1276499107,2cyclictest14536-21wget08:20:150
127739998,1cyclictest16163-21chrt08:20:423
127719993,4cyclictest3875-21tr09:05:152
127649996,2cyclictest30369-21grep07:45:160
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional