You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-12 - 21:46

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Tue May 12, 2026 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2389729482,8sleep00-21swapper/007:07:000
2296029466,9sleep20-21swapper/207:05:222
2408528359,8sleep30-21swapper/307:09:293
2399628164,13sleep10-21swapper/107:08:181
2421499290,28cyclictest0-21swapper/209:15:502
2421499280,27cyclictest0-21swapper/210:05:552
2420299240,22cyclictest0-21swapper/009:13:110
2421499230,1cyclictest0-21swapper/209:13:112
2420299230,22cyclictest0-21swapper/011:20:530
24219992211,9cyclictest4411-21idleruntime-cro10:20:003
2421499220,22cyclictest0-21swapper/211:51:362
2421499220,22cyclictest0-21swapper/211:20:532
2421099220,4cyclictest16861-21diskmemload10:59:051
2420299220,21cyclictest0-21swapper/011:51:360
2421499210,21cyclictest0-21swapper/209:45:562
2421099218,1cyclictest12453-21ssh10:44:001
24210992119,2cyclictest16861-21diskmemload11:40:571
2421999202,4cyclictest0-21swapper/310:29:103
2421499201,2cyclictest24835-21apt-config11:50:012
2421499200,3cyclictest0-21swapper/208:10:132
24202992018,1cyclictest0-21swapper/011:25:040
2421099192,17cyclictest16861-21diskmemload10:45:381
24210991917,2cyclictest25-21ksoftirqd/110:09:061
24210991917,1cyclictest0-21swapper/111:36:221
2421099190,18cyclictest0-21swapper/111:19:551
2420299190,3cyclictest0-21swapper/011:31:160
2420299190,2cyclictest0-21swapper/010:12:520
2420299190,16cyclictest0-21swapper/009:19:070
2421999182,3cyclictest141rcu_preempt10:49:243
2421999182,1cyclictest0-21swapper/311:52:353
24219991813,2cyclictest225-21systemd-journal09:55:373
2421499180,17cyclictest0-21swapper/211:25:042
2421499180,17cyclictest0-21swapper/210:30:032
2421099181,1cyclictest141rcu_preempt10:52:221
24202991817,1cyclictest0-21swapper/010:51:100
2421499171,15cyclictest0-21swapper/212:25:122
24214991710,7cyclictest0-21swapper/210:11:532
24202991715,1cyclictest0-21swapper/012:14:240
2420299170,1cyclictest0-21swapper/010:08:330
2420299170,1cyclictest0-21swapper/009:45:560
24219991611,2cyclictest16861-21diskmemload10:35:503
2421499165,6cyclictest6931-21sh11:00:142
2421499162,6cyclictest0-21swapper/208:50:142
24214991614,1cyclictest0-21swapper/211:13:182
2421499160,15cyclictest0-21swapper/210:48:402
24210991615,1cyclictest31149-21ssh09:18:061
24210991613,2cyclictest0-21swapper/109:59:271
2421099160,15cyclictest16861-21diskmemload09:20:571
2420299166,9cyclictest24456-21apt-get07:10:150
2420299160,16cyclictest0-21swapper/011:12:180
2421999153,4cyclictest141rcu_preempt09:11:533
2421999152,8cyclictest11139-21apt-get07:50:153
2421499150,15cyclictest0-21swapper/211:55:402
2421499150,14cyclictest0-21swapper/210:15:112
24210991514,1cyclictest0-21swapper/110:28:591
24210991513,1cyclictest16861-21diskmemload11:47:561
24210991512,2cyclictest241rcuc/107:45:271
2420299158,5cyclictest31399-21cstates10:55:150
2420299154,10cyclictest12416-21ssh11:41:490
2420299151,6cyclictest15604-21cron08:00:000
2420299150,14cyclictest0-21swapper/009:33:580
2421999141,8cyclictest0-21swapper/312:14:053
2421999141,12cyclictest17221-21find08:00:233
24219991410,1cyclictest39-21ksoftirqd/311:17:573
2421999140,2cyclictest0-21swapper/311:25:263
2421499149,4cyclictest32214-21cut11:15:152
2421499147,6cyclictest26238-21sh09:54:042
2421499141,7cyclictest0-21swapper/211:09:382
2421499140,1cyclictest0-21swapper/209:42:432
2421099145,4cyclictest537-21snmpd11:14:371
2421099144,9cyclictest917-21munin-run11:55:001
24210991414,0cyclictest0-21swapper/112:35:361
2421099141,3cyclictest16861-21diskmemload12:18:381
24210991410,4cyclictest0-21swapper/110:30:111
2421099140,9cyclictest0-21swapper/107:55:121
2421099140,7cyclictest0-21swapper/112:06:071
2421099140,1cyclictest0-21swapper/109:29:431
2421099140,1cyclictest0-21swapper/109:13:481
2420299146,6cyclictest8774-21sh10:21:540
2420299144,9cyclictest10932-21hddtemp_smartct09:25:170
2420299140,3cyclictest0-21swapper/012:06:520
2421999139,3cyclictest12558-21tr10:05:163
2421999139,3cyclictest10113-21grep09:25:143
2421999137,6cyclictest22903-21kworker/u8:011:58:283
2421999135,6cyclictest16861-21diskmemload10:23:503
2421999133,3cyclictest18439-21apt-get09:30:143
2421999133,3cyclictest0-21swapper/311:02:183
2421999132,3cyclictest1626-21apt-get07:30:153
2421999131,5cyclictest3294-21/usr/sbin/munin09:40:153
24219991311,1cyclictest16861-21diskmemload09:48:353
24219991310,3cyclictest39-21ksoftirqd/312:35:143
2421999130,9cyclictest0-21swapper/309:38:483
2421999130,13cyclictest0-21swapper/312:15:583
24214991312,1cyclictest0-21swapper/211:43:492
24214991310,2cyclictest23418-21grep10:50:182
2421499130,12cyclictest0-21swapper/210:29:162
2421499130,0cyclictest0-21swapper/212:14:242
2421099139,4cyclictest38150irq/131-snd_hda09:34:551
2421099139,2cyclictest32447-21grep07:25:191
2421099139,2cyclictest25507-21cat07:10:201
2421099133,1cyclictest16861-21diskmemload12:28:471
24210991312,1cyclictest17430-21ssh11:05:361
24210991310,2cyclictest29849-21grep07:20:181
24210991310,2cyclictest25-21ksoftirqd/110:15:001
24210991310,2cyclictest18046-21wc08:05:001
2420299137,3cyclictest13-21ksoftirqd/010:15:130
24202991312,1cyclictest0-21swapper/009:24:530
24202991310,2cyclictest2968-21awk12:35:000
2420299130,9cyclictest0-21swapper/009:56:090
2420299130,11cyclictest0-21swapper/009:54:110
51332120,0chrt0-21swapper/011:56:470
2421999129,2cyclictest8733-21cut11:40:143
2421999129,2cyclictest6594-21grep10:40:173
2421999129,2cyclictest28360-21cut12:30:163
2421999129,2cyclictest26134-21wc11:30:253
2421999129,2cyclictest22263-21sed08:10:263
2421999129,2cyclictest20996-21sh11:47:023
2421999129,2cyclictest15607-21cron08:00:003
2421999128,3cyclictest18670-21sh12:25:003
2421999126,5cyclictest31309-21apache210:57:513
24219991210,1cyclictest429-21tr08:35:133
2421999120,7cyclictest0-21swapper/310:31:513
2421999120,11cyclictest18986-21cut12:05:193
2421499129,2cyclictest8088-21grep07:40:282
2421499129,2cyclictest5646-21ssh10:40:022
2421499129,2cyclictest5483-21ssh11:37:562
2421499129,2cyclictest5330-21cut10:00:232
2421499129,2cyclictest2955-21dump-pmu-power12:35:002
2421499129,2cyclictest29199-21grep09:55:202
2421499129,2cyclictest26171-21tail08:20:152
2421499129,2cyclictest2578-21readlink08:40:012
2421499129,2cyclictest19377-21grep08:05:192
2421499129,2cyclictest1726-21taskset09:20:122
2421499129,2cyclictest16985-21cat08:00:202
2421499129,2cyclictest1695-21sh10:56:192
2421499129,2cyclictest12770-21ssh12:01:132
2421499129,2cyclictest12058-21grep07:50:202
2421499128,2cyclictest0-21swapper/207:25:162
2421499122,6cyclictest16861-21diskmemload09:31:252
2421499122,6cyclictest10981-21sshd12:39:372
2421499121,9cyclictest26773-21apache_volume09:35:132
24214991210,1cyclictest5770-21cut10:20:162
2421499120,12cyclictest0-21swapper/210:35:212
2421099129,2cyclictest930-21sh12:33:251
2421099129,2cyclictest31619-21smart_sda10:35:261
2421099129,2cyclictest1696-21ssh12:14:561
2421099128,4cyclictest38150irq/131-snd_hda10:19:121
2421099126,6cyclictest537-21snmpd09:37:311
24210991211,1cyclictest25680-21tr08:20:131
2421099121,10cyclictest6253-21ntpq10:20:211
24210991210,1cyclictest0-21swapper/109:45:201
2421099120,1cyclictest0-21swapper/109:44:071
2421099120,11cyclictest21428-21grep08:10:181
2420299129,2cyclictest996-21mailstats11:15:230
2420299129,2cyclictest832-21ssh10:36:270
2420299129,2cyclictest8039-21expr08:50:160
2420299129,2cyclictest7855-21cat11:00:190
2420299129,2cyclictest6248-21date07:40:000
2420299129,2cyclictest29855-21grep07:20:180
2420299129,2cyclictest27898-21ssh10:33:500
2420299129,2cyclictest16641-21rm10:45:450
24202991210,1cyclictest19876-21grep12:25:150
2420299120,6cyclictest0-21swapper/009:39:110
2420299120,11cyclictest10255-21ssh12:00:150
2421999119,2cyclictest21661-21ssh10:10:203
2421999119,1cyclictest29804-21sh07:20:173
2421999119,1cyclictest272842sleep308:21:483
2421999118,2cyclictest9153-21ssh11:20:223
2421999118,2cyclictest7881-21grep08:50:163
2421999118,2cyclictest7008-21chrt08:46:043
2421999118,2cyclictest6458-21cut07:40:143
2421999118,2cyclictest6153-21wc07:40:003
2421999118,2cyclictest39-21ksoftirqd/309:50:223
2421999118,2cyclictest3514-21taskset11:36:223
2421999118,2cyclictest32420-21sed07:25:193
2421999118,2cyclictest27527-21cut07:15:163
2421999118,2cyclictest25793-21cut09:15:153
2421999118,2cyclictest25413-21date11:10:273
2421999118,2cyclictest2359-21ssh09:20:163
2421999118,2cyclictest18285-21apt-get08:05:133
2421999118,2cyclictest16729-21chrt09:08:403
2421999118,2cyclictest15642-21ls11:05:163
2421999118,2cyclictest14183-21chrt09:01:313
2421999118,2cyclictest11558-21cat12:00:263
2421999110,10cyclictest4202-21cat10:00:163
2421999110,10cyclictest23280-21ssh12:27:053
2421499119,2cyclictest30331-21grep07:20:222
2421499119,2cyclictest14063-21head09:00:282
2421499119,2cyclictest10563-21fschecks_count08:55:162
2421499119,1cyclictest9892-21tr07:45:222
2421499119,1cyclictest6031-21seq07:39:182
2421499119,1cyclictest31793-21sort08:30:242
2421499119,1cyclictest29251-21wc08:25:252
2421499119,1cyclictest12962-21ssh12:20:292
2421499118,2cyclictest27250-21chrt11:31:082
2421499118,2cyclictest11258-21cat09:25:192
2421499118,2cyclictest10343-21apt-config12:20:002
2421499110,10cyclictest24288-21ssh12:08:432
2421099119,2cyclictest8396-21ssh10:02:221
2421099119,2cyclictest4393-21rm11:56:121
2421099119,2cyclictest1233-21cat08:35:181
2421099119,1cyclictest24747-21cat11:30:141
2421099119,1cyclictest17914-21ssh11:25:261
2421099119,1cyclictest16359-21grep09:05:281
2421099119,1cyclictest10714-21sh12:20:051
2421099118,2cyclictest9590-21cat11:20:271
2421099118,2cyclictest7395-21cat08:50:011
2421099118,2cyclictest6629-21sh11:00:021
2421099118,2cyclictest28055-21cut08:25:151
2421099118,2cyclictest27573-21gltestperf07:15:171
2421099118,2cyclictest19369-21chrt09:50:011
2421099118,2cyclictest13247-21munin-plugin-st07:55:001
2421099118,2cyclictest11211-21wc12:00:231
2421099114,4cyclictest2338-21cron08:44:551
24210991110,1cyclictest19051-21tail08:05:171
2421099110,9cyclictest0-21swapper/108:15:201
2420299119,2cyclictest7074-21taskset08:46:540
2420299119,2cyclictest6254-21ssh09:41:370
2420299119,2cyclictest13498-21cpuspeed_turbos10:25:140
2420299119,1cyclictest61882sleep012:35:500
2420299119,1cyclictest423-21tr08:35:130
2420299119,1cyclictest28586-21grep08:25:170
2420299119,1cyclictest26151-21grep08:20:150
2420299119,1cyclictest23380-21ls08:15:150
2420299119,1cyclictest15889-21basename08:00:120
2420299119,1cyclictest15021-21expr09:05:140
2420299119,1cyclictest13043-21taskset07:52:370
2420299119,1cyclictest12367-21dpkg09:00:140
2420299119,1cyclictest10201-21tr08:55:140
2420299118,2cyclictest9838-21ssh10:41:580
2420299118,2cyclictest813-21ssh11:35:170
2420299118,2cyclictest7415-21cat07:40:200
2420299118,2cyclictest6036-21ssh12:16:480
2420299118,2cyclictest3163-21expr08:40:150
2420299118,2cyclictest26788-21taskset07:15:020
2420299118,2cyclictest20900-21tr08:10:150
2420299118,2cyclictest18865-21grep11:45:270
2420299118,2cyclictest18269-21bc08:05:110
2420299118,2cyclictest16030-21head11:05:190
2420299118,2cyclictest15061-21ssh12:22:090
223932110,0sleep30-21swapper/310:50:073
2421999108,1cyclictest30271-21cut08:30:123
2421999108,1cyclictest229902sleep308:15:073
2421999107,2cyclictest9045-21grep07:45:163
2421999107,2cyclictest4282-21sed08:40:253
2421999107,2cyclictest292042sleep308:25:253
2421999107,2cyclictest24798-21cat07:10:153
2421999107,2cyclictest11062-21grep08:55:193
2421499109,1cyclictest23597-21sed08:15:162
2421499108,2cyclictest7287-21chrt08:49:402
2421499108,2cyclictest16606-21taskset09:07:052
2421499108,1cyclictest4355-21wc08:40:252
2421499108,1cyclictest13632-21basename07:55:132
2421499103,3cyclictest0-21swapper/207:10:142
2421099108,2cyclictest3541-21taskset07:32:451
2421099108,1cyclictest6456-21wget07:40:141
2421099108,1cyclictest30458-21tr08:30:121
2421099108,1cyclictest13881-21cat09:00:281
2421099104,3cyclictest9579-21sshd08:53:371
2421099101,2cyclictest0-21swapper/107:35:141
2420299109,1cyclictest2022-21expr07:30:170
2420299108,2cyclictest3952-21ssh10:00:140
2420299108,1cyclictest8819-21aten2_r7power_e07:45:150
2420299108,1cyclictest31727-21basename07:25:160
2420299108,1cyclictest30301-21basename08:30:120
242149998,1cyclictest3174-21sed07:30:282
242149997,1cyclictest27932-21grep07:15:192
242109997,1cyclictest10326-21expr08:55:151
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional