You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-26 - 16:56

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot7s.osadl.org (updated Fri Dec 26, 2025 12:43:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
690429580,10sleep00-21swapper/007:06:290
691129166,9sleep20-21swapper/207:06:362
664528964,3sleep10-21swapper/107:05:311
713628471,9sleep30-21swapper/307:09:303
127602710,0sleep00-21swapper/011:50:170
726699280,25cyclictest0-21swapper/111:35:151
726699220,2cyclictest28655-21grep11:15:191
7270992119,1cyclictest0-21swapper/211:30:122
727099210,19cyclictest0-21swapper/211:45:112
727099210,19cyclictest0-21swapper/208:25:142
726499211,17cyclictest0-21swapper/009:30:130
727299203,11cyclictest0-21swapper/312:00:133
727299200,1cyclictest0-21swapper/308:50:143
7266991917,1cyclictest30172-21aten2_r7power_p09:05:151
7264991918,1cyclictest0-21swapper/011:30:120
727099181,16cyclictest19551-21ntp_states10:55:202
726699180,18cyclictest0-21swapper/111:10:161
726699169,6cyclictest16730-21grep10:50:171
726499163,5cyclictest0-21swapper/008:15:150
726499160,3cyclictest0-21swapper/008:25:140
727299150,7cyclictest0-21swapper/307:20:133
727099150,14cyclictest0-21swapper/207:20:192
726699158,6cyclictest31193-21cron12:30:001
727299147,6cyclictest6284-21apt-get08:15:153
7272991413,1cyclictest0-21swapper/309:55:003
727099149,2cyclictest17867-21chrt08:39:352
726499147,6cyclictest26441-21munin-run12:20:000
727299137,5cyclictest27650-21sh11:15:003
7272991311,2cyclictest39-21ksoftirqd/307:55:253
7272991310,2cyclictest13172-21grep11:50:193
727099131,2cyclictest55550irq/126-enp2s0-09:39:222
7270991310,2cyclictest30736-21sort09:05:182
7270991310,2cyclictest18235-21sed07:30:192
7270991310,2cyclictest14627-21sed08:30:212
726699134,7cyclictest25-21ksoftirqd/107:50:141
7266991312,1cyclictest25-21ksoftirqd/109:30:261
7266991310,2cyclictest29851-21tail10:10:171
7266991310,1cyclictest0-21swapper/110:20:151
726499136,6cyclictest14395-21munin-run09:40:010
7264991310,2cyclictest25940-21grep08:55:170
727299129,2cyclictest9943-21cut07:15:133
727299129,2cyclictest8621-21tr11:40:243
727299129,2cyclictest5828-21grep09:20:183
727299129,2cyclictest5577-21tail08:10:253
727299129,2cyclictest32429-21chrt09:10:103
727299129,2cyclictest32429-21chrt09:10:093
727299129,2cyclictest31050-21grep11:20:183
727299129,2cyclictest30846-21grep10:10:273
727299129,2cyclictest30023-21cron11:20:003
727299129,2cyclictest28772-21cat10:10:003
727299129,2cyclictest19501-21grep07:35:163
727299129,2cyclictest18833-21cut08:40:183
727299129,2cyclictest16038-21seq07:25:253
727299129,2cyclictest12148-21cut09:35:153
727299129,2cyclictest1144-21sh12:35:003
727299129,2cyclictest10382-21head08:20:273
727299128,3cyclictest21513-21grep11:00:183
7272991210,1cyclictest29742-21cut12:25:173
7272991210,1cyclictest25385-21grep08:55:153
7272991210,1cyclictest22699-21basename08:50:003
7272991210,1cyclictest10256-21cut09:30:163
727099129,2cyclictest7743-21tr07:10:142
727099129,2cyclictest7743-21tr07:10:142
727099129,2cyclictest29037-21hostname07:55:152
727099129,2cyclictest27968-21tr09:00:142
727099129,2cyclictest18927-21grep08:40:182
727099129,2cyclictest14535-21cron11:54:592
727099129,2cyclictest11356-21grep07:15:262
7270991210,1cyclictest25043-21cron08:55:002
727099120,3cyclictest0-21swapper/211:05:112
727099120,3cyclictest0-21swapper/208:15:152
727099120,11cyclictest6729-21tr10:30:152
727099120,11cyclictest19307-21sh12:05:002
727099120,11cyclictest0-21swapper/211:00:142
726699129,2cyclictest6991-21apt09:25:011
726699129,2cyclictest4643-21head08:10:161
726699129,2cyclictest4581-21cron09:20:001
726699129,2cyclictest31919-21tr10:15:151
726699129,2cyclictest31689-21grep12:30:161
726699129,2cyclictest20345-21grep12:05:161
726699129,2cyclictest15100-21grep08:30:251
726699128,3cyclictest18876-21taskset09:46:451
726699122,9cyclictest6048-21kworker/1:210:35:141
7266991210,1cyclictest19680-21cat08:40:271
7266991210,1cyclictest15098-21grep11:55:151
726699120,8cyclictest38150irq/131-snd_hda12:00:121
726699120,11cyclictest9133-21sed07:10:261
726699120,11cyclictest9133-21sed07:10:261
726699120,11cyclictest16424-21cut08:35:171
726499129,2cyclictest8859-21basename10:35:120
726499129,2cyclictest7768-21grep11:40:180
726499129,2cyclictest3778-21taskset12:39:530
726499129,2cyclictest32424-21munin-run11:25:000
726499129,2cyclictest3116-21grep08:05:260
726499129,2cyclictest29225-21expr07:55:160
726499129,2cyclictest26570-21aten2_r7power_c10:05:140
726499129,2cyclictest23226-21tr08:50:150
726499129,2cyclictest23014-21grep09:55:220
726499129,2cyclictest17729-21grep07:30:140
726499129,2cyclictest16031-21grep11:55:220
726499129,2cyclictest14370-21cron07:25:000
726499122,1cyclictest225-21systemd-journal07:40:000
7264991210,1cyclictest28376-21cut09:00:170
7264991210,1cyclictest1654-21grep09:10:240
7264991210,1cyclictest1654-21grep09:10:240
727299119,2cyclictest18065-21sort09:45:193
727299119,1cyclictest25215-21wc07:45:193
727299119,1cyclictest23611-21chrt09:56:373
727299119,1cyclictest21945-21chrt12:10:113
727299119,1cyclictest19549-21grep10:55:193
727299119,1cyclictest16956-21ntpq08:35:213
727299119,1cyclictest16740-21seq11:57:543
727299119,1cyclictest15228-21chrt08:31:113
727299119,1cyclictest11238-21taskset10:40:033
727299118,2cyclictest968-21cron08:04:593
727299118,2cyclictest7486-21cat10:30:193
727299118,2cyclictest5233-21grep10:25:203
727299118,2cyclictest4471-21seq11:30:503
727299118,2cyclictest3714-21chrt10:23:133
727299118,2cyclictest31737-21tail09:05:273
727299118,2cyclictest27933-21tr09:00:143
727299118,2cyclictest26991-21tr12:20:143
727299118,2cyclictest26644-21awk07:50:153
727299118,2cyclictest26444-21sh12:20:003
727299118,2cyclictest24220-21wc11:05:173
727299118,2cyclictest20805-21tr12:05:213
727299118,2cyclictest1902-21tr12:35:123
727299118,2cyclictest18669-21sed07:30:233
727299118,2cyclictest14747-21cut10:45:193
727299118,2cyclictest10144-21cut11:45:123
727299110,9cyclictest0-21swapper/310:15:143
727299110,10cyclictest9536-21cut10:35:183
727299110,10cyclictest22099-21wc07:40:143
727099119,2cyclictest8601-21ntpq11:40:232
727099119,2cyclictest3250-21grep12:35:232
727099119,2cyclictest30204-21sed10:10:212
727099119,2cyclictest2083-21chrt11:26:232
727099119,1cyclictest71272sleep211:39:302
727099119,1cyclictest32439-21grep08:00:212
727099119,1cyclictest3189-21sed09:15:182
727099119,1cyclictest24404-21tr10:00:162
727099119,1cyclictest236772sleep207:42:212
727099119,1cyclictest22053-21grep12:10:142
727099119,1cyclictest21709-21wc07:40:012
727099118,2cyclictest8925-21tr08:20:152
727099118,2cyclictest4965-21grep09:20:132
727099118,2cyclictest32456-21wc12:30:202
727099118,2cyclictest28703-21cut11:15:192
727099118,2cyclictest27417-21cron09:00:002
727099118,2cyclictest27305-21tail07:50:182
727099118,2cyclictest22219-21grep08:45:262
727099118,2cyclictest16648-21chrt09:43:382
727099118,2cyclictest1345-21chrt10:19:202
727099118,2cyclictest11006-21ls10:40:002
726699119,2cyclictest32263-21taskset11:23:101
726699119,2cyclictest11336-21grep10:40:151
726699119,1cyclictest9968-21basename07:15:151
726699119,1cyclictest8611-21cron10:35:001
726699119,1cyclictest7625-21egrep11:40:171
726699119,1cyclictest32731-21grep09:10:121
726699119,1cyclictest32731-21grep09:10:121
726699119,1cyclictest28284-21taskset09:00:151
726699119,1cyclictest26776-21aten2_r7power_e12:20:131
726699119,1cyclictest24930-21head07:45:161
726699119,1cyclictest24734-21expr12:15:141
726699119,1cyclictest23401-21expr11:05:121
726699119,1cyclictest22378-21expr12:10:151
726699119,1cyclictest21136-21cat11:00:161
726699119,1cyclictest15028-21expr07:25:151
726699119,1cyclictest12940-21uname09:35:191
726699119,1cyclictest12447-21aten2_r7power_c11:50:161
726699119,1cyclictest11299-21tr08:25:161
726699118,3cyclictest14083-21seq07:21:201
726699118,2cyclictest9792-21cut08:20:211
726699118,2cyclictest7973-21head09:25:171
726699118,2cyclictest512-21tr11:25:161
726699118,2cyclictest4558-21expr10:25:141
726699118,2cyclictest2919-21expr11:30:131
726699116,4cyclictest25542-21expr08:55:151
726699110,10cyclictest26516-21/usr/sbin/munin10:05:121
726499119,1cyclictest8151-21cut07:10:160
726499119,1cyclictest8151-21cut07:10:160
726499119,1cyclictest7297-21sed10:30:170
726499119,1cyclictest70112sleep011:37:590
726499119,1cyclictest30804-21grep12:25:270
726499119,1cyclictest29745-21grep10:10:170
726499119,1cyclictest286192sleep007:54:530
726499119,1cyclictest24968-21taskset11:06:140
726499119,1cyclictest24444-21grep10:00:150
726499119,1cyclictest22238-21expr07:40:140
726499119,1cyclictest20808-21cut11:00:140
726499119,1cyclictest19362-21chrt09:50:010
726499119,1cyclictest17589-21taskset08:36:000
726499119,1cyclictest17563-21expr12:00:140
726499119,1cyclictest15293-21cut07:25:160
726499119,1cyclictest13876-21expr08:30:140
726499118,2cyclictest7663-21expr09:25:150
726499118,2cyclictest5608-21sed08:10:250
726499118,2cyclictest32183-21expr12:30:180
726499118,2cyclictest27099-21expr12:20:140
726499118,2cyclictest2565-21egrep09:15:150
726499118,2cyclictest23211-21wc12:10:210
726499118,2cyclictest20675-21sed12:05:190
726499118,2cyclictest19027-21taskset09:48:430
726499118,2cyclictest18582-21expr08:40:160
726499118,2cyclictest16949-21date09:45:010
726499118,2cyclictest15731-21chrt10:49:470
726499118,2cyclictest11355-21cat07:15:270
726499116,2cyclictest225-21systemd-journal08:04:590
726499112,7cyclictest29179-21grep11:15:230
726499110,10cyclictest24454-21ls07:45:140
727299109,1cyclictest8965-21chrt07:10:253
727299109,1cyclictest8965-21chrt07:10:253
727299108,1cyclictest2186-21chrt11:27:443
727299107,2cyclictest8024-21tail09:25:173
727299107,2cyclictest2705-21tr09:15:153
727299107,2cyclictest26071-21chrt10:02:263
727299107,2cyclictest16365-21grep09:40:283
727299107,2cyclictest15993-21taskset10:50:093
727299107,2cyclictest11090-21cut08:25:153
727299106,3cyclictest7256-21cyclictest08:05:013
727299105,2cyclictest225-21systemd-journal11:40:003
727099109,1cyclictest3703-21chrt10:23:042
727099109,1cyclictest30946-21chrt12:26:532
727099109,1cyclictest26748-21cut12:20:132
727099109,1cyclictest24842-21cut07:45:162
727099109,1cyclictest169902sleep209:45:022
727099109,1cyclictest15255-21expr11:55:152
727099108,2cyclictest25555-21awk12:15:222
727099108,2cyclictest16396-21chrt07:25:352
727099108,2cyclictest10304-21cut09:30:172
727099108,1cyclictest5064-21ntp_states08:10:202
727099108,1cyclictest32727-21tr09:10:132
727099108,1cyclictest32727-21tr09:10:122
727099108,1cyclictest25816-21grep11:10:132
727099108,1cyclictest155172sleep210:47:002
727099107,2cyclictest5834-21sed10:25:252
727099107,2cyclictest26816-21expr10:05:142
727099107,2cyclictest2503-21wc08:05:202
727099107,2cyclictest22767-21wc09:55:192
727099107,2cyclictest20322-21grep12:05:162
727099107,2cyclictest20163-21cut09:50:172
727099107,2cyclictest17265-21tr10:50:222
727099107,2cyclictest12067-21sed10:40:192
727099106,3cyclictest30315-21apache_volume11:20:132
727099101,5cyclictest0-21swapper/209:25:132
726699109,1cyclictest7492sleep108:02:161
726699109,1cyclictest23899-21latency08:50:191
726699109,1cyclictest23692-21taskset07:42:341
726699109,1cyclictest200262sleep110:55:261
726699109,1cyclictest17260-21expr07:30:121
726699109,1cyclictest16429-21seq09:40:471
726699108,1cyclictest24376-21cat10:00:151
726699108,1cyclictest21877-21hostname09:55:141
726699108,1cyclictest19732-21expr07:35:161
726699108,1cyclictest19451-21basename09:50:131
726699108,1cyclictest13926-21tr10:45:141
726699108,1cyclictest10283-21cut11:45:121
726699107,2cyclictest6543-21grep08:15:161
726699107,2cyclictest21805-21grep08:45:201
726699107,2cyclictest1926-21expr08:05:151
726699105,3cyclictest1637-21apt-get12:35:101
726499109,1cyclictest21028-21expr08:45:130
726499109,1cyclictest18820-21expr10:55:140
726499108,1cyclictest9987-21cat11:45:010
726499108,1cyclictest5104-21grep09:20:130
726499108,1cyclictest322-21wget11:25:150
726499108,1cyclictest30277-21cpuspeed_turbos09:05:150
726499108,1cyclictest25500-21taskset11:10:030
726499108,1cyclictest11349-21wget10:40:150
726499108,1cyclictest0-21swapper/008:20:140
726499107,2cyclictest4913-21grep10:25:160
726499107,2cyclictest32457-21grep10:15:190
726499101,7cyclictest1702-21apache_volume10:20:140
72669997,1cyclictest29050-21egrep07:55:151
72649991,6cyclictest0-21swapper/010:50:140
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional