You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-29 - 06:11

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Mon Dec 29, 2025 00:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1209429178,9sleep20-21swapper/219:09:382
1199829063,9sleep00-21swapper/019:08:220
1104128766,16sleep30-21swapper/319:05:213
1191228265,13sleep10-21swapper/119:07:161
1221599231,0cyclictest0-21swapper/023:50:170
1221899214,12cyclictest0-21swapper/119:25:131
1221599200,3cyclictest0-21swapper/020:45:150
1222299190,3cyclictest0-21swapper/200:10:162
1221899183,12cyclictest0-21swapper/122:35:131
12215991817,1cyclictest0-21swapper/020:15:160
1222499170,1cyclictest0-21swapper/323:30:233
1221899173,1cyclictest141rcu_preempt20:45:271
12218991713,3cyclictest25-21ksoftirqd/123:10:141
1222499168,2cyclictest141rcu_preempt19:50:233
1222499168,1cyclictest141rcu_preempt20:45:223
1222499166,3cyclictest0-21swapper/321:35:143
1221899168,2cyclictest141rcu_preempt23:05:151
1222499153,5cyclictest0-21swapper/319:55:163
12224991513,1cyclictest0-21swapper/300:40:013
1222299156,5cyclictest12209-21cyclictest19:10:212
1222299156,5cyclictest12209-21cyclictest19:10:212
1222299150,3cyclictest225-21systemd-journal20:34:542
12215991510,3cyclictest23373-21cron20:40:010
1222499147,3cyclictest381rcuc/319:10:223
1222499147,3cyclictest381rcuc/319:10:213
1222499143,2cyclictest141rcu_preempt22:50:093
12224991411,2cyclictest39-21ksoftirqd/321:20:003
12224991411,2cyclictest20956-21cron22:50:003
1222499140,3cyclictest39-21ksoftirqd/323:05:133
1222299149,1cyclictest32-21ksoftirqd/221:05:172
12222991413,1cyclictest291irq_work/222:20:122
12222991411,2cyclictest3598-21wc00:25:262
12222991411,2cyclictest32-21ksoftirqd/219:35:102
12222991411,2cyclictest18762-21wc20:30:002
12222991410,3cyclictest16566-21wc23:45:292
12222991410,3cyclictest12464-21wc23:40:012
1222299140,1cyclictest0-21swapper/221:45:162
12218991411,2cyclictest24135-21perf19:35:011
1221899140,3cyclictest225-21systemd-journal23:02:081
12215991410,3cyclictest2465-21sed21:00:300
1221599140,13cyclictest0-21swapper/000:20:150
1222499139,3cyclictest39-21ksoftirqd/322:05:003
1222499139,3cyclictest12327-21cron23:40:013
1222499138,4cyclictest39-21ksoftirqd/322:10:003
12224991310,2cyclictest3377-21tr23:20:173
12224991310,2cyclictest18634-21wc19:20:243
12224991310,2cyclictest11385-21mktemp22:30:013
1222299139,3cyclictest3474-21chrt19:55:292
1222299139,3cyclictest2661-21tail22:10:192
1222299136,2cyclictest32-21ksoftirqd/223:00:012
12222991311,1cyclictest32-21ksoftirqd/200:39:012
12222991310,3cyclictest32-21ksoftirqd/221:40:112
12222991310,2cyclictest5803-21wc22:15:272
12222991310,2cyclictest30416-21tune2fs00:15:172
12222991310,2cyclictest26662-21cut20:45:192
12222991310,2cyclictest21825-21cat19:30:012
12222991310,2cyclictest20998-21wc23:55:192
12222991310,2cyclictest11219-21tail21:20:182
12222991310,2cyclictest10779-21grep22:25:262
1222299130,12cyclictest5754-21tr23:25:142
1221899139,3cyclictest18522-21cron22:45:001
1221899134,8cyclictest38150irq/131-snd_hda00:10:161
12218991310,2cyclictest5195-21cron23:25:001
12218991310,2cyclictest30644-21cat22:00:201
12218991310,2cyclictest19604-21head20:30:161
12218991310,2cyclictest18565-21cron20:29:591
1221899130,12cyclictest0-21swapper/121:00:161
1221899130,11cyclictest28479-21tr20:50:131
1221599136,6cyclictest9967-21munin-run23:35:000
1221599135,7cyclictest22122-21munin-run23:59:590
1221599131,6cyclictest2854-21sendmail-msp23:20:000
12215991310,2cyclictest31980-21grep23:10:240
12215991310,2cyclictest28885-21cron19:45:000
12215991310,2cyclictest23388-21cat00:00:180
12215991310,2cyclictest22175-21cron21:45:000
12215991310,2cyclictest1655-21apt22:10:000
12215991310,2cyclictest14540-21cron19:15:010
12215991310,2cyclictest14540-21cron19:15:000
1221599130,12cyclictest25114-21cat22:55:310
1222499129,3cyclictest39-21ksoftirqd/321:40:203
1222499129,2cyclictest5912-21sed22:15:273
1222499129,2cyclictest5367-21cat21:10:003
1222499129,2cyclictest30011-21grep20:50:243
1222499129,2cyclictest24992-21cut19:35:153
1222499129,2cyclictest21653-21grep23:55:263
1222499129,2cyclictest19485-21munin-run23:55:003
1222499129,2cyclictest17115-21cut20:25:163
1222499129,2cyclictest15537-21tr21:30:143
1222499129,2cyclictest13169-21grep23:40:163
1222499128,3cyclictest2885-21tail00:25:183
1222499128,3cyclictest27416-21grep21:55:143
1222499128,3cyclictest23236-21ls19:30:213
1222499127,4cyclictest39-21ksoftirqd/300:20:183
1222499126,2cyclictest17183-21head22:40:183
1222499125,6cyclictest21273-21unixbench-2d19:25:263
1222499123,6cyclictest10603-21cut21:20:153
12224991210,1cyclictest39-21ksoftirqd/320:00:133
1222299129,2cyclictest640-21wc19:50:232
1222299129,2cyclictest21902-21cut22:50:142
1222299129,2cyclictest19517-21grep22:45:172
1222299129,2cyclictest14978-21cron21:30:002
1222299129,2cyclictest11109-21sort20:14:592
1222299128,3cyclictest29512-21head20:50:182
1222299126,5cyclictest32-21ksoftirqd/220:20:192
1222299124,7cyclictest19807-21apache220:35:182
1222299121,0cyclictest0-21swapper/223:50:172
1222299120,11cyclictest6627-21cron00:35:002
1222299120,11cyclictest30309-21head22:00:182
1221899129,2cyclictest939-21sed19:50:261
1221899129,2cyclictest8960-21grep20:10:161
1221899129,2cyclictest5226-21cut00:30:181
1221899129,2cyclictest2292-21tr00:25:151
1221899129,2cyclictest21621-21grep23:55:261
1221899129,2cyclictest13424-21lockfile-create20:19:591
1221899124,4cyclictest225-21systemd-journal00:19:591
12218991210,1cyclictest7456-21head22:20:161
12218991210,1cyclictest28909-21grep21:55:261
12218991210,1cyclictest14163-21cut20:20:181
12218991210,1cyclictest10225-21turbostat21:20:001
1221899120,3cyclictest0-21swapper/123:45:161
1221899120,1cyclictest0-21swapper/121:25:241
1221899120,11cyclictest16944-21grep19:20:011
1221599129,2cyclictest7718-21cat22:20:190
1221599129,2cyclictest6756-21cat21:10:220
1221599129,2cyclictest6756-21cat21:10:210
1221599129,2cyclictest5208-21sort00:30:180
1221599129,2cyclictest31407-21grep20:55:180
1221599129,2cyclictest28969-21cat00:10:280
1221599129,2cyclictest27440-21grep21:55:140
1221599129,2cyclictest25574-21head21:50:190
1221599129,2cyclictest18818-21ls21:35:230
1221599129,2cyclictest11147-21grep21:20:170
1221599125,3cyclictest225-21systemd-journal23:10:000
1222499118,2cyclictest8655-21wc20:10:003
1222499118,2cyclictest5785-21wc00:30:233
1222499118,2cyclictest2822-21munin-run23:20:003
1222499118,2cyclictest28112-21awk23:05:003
1222499118,2cyclictest27087-21wc19:40:183
1222499118,2cyclictest25592-21sed21:50:183
1222499118,2cyclictest25058-21grep00:05:143
1222499118,2cyclictest25058-21grep00:05:143
1222499118,2cyclictest23893-21tr20:40:143
1222499118,2cyclictest1506-21tr21:00:203
1222499118,2cyclictest14758-21tail22:35:173
1222499118,2cyclictest14573-21sed21:25:283
1222499117,3cyclictest381rcuc/322:10:183
1222499117,3cyclictest24075-21sed21:45:293
1222499117,3cyclictest19488-21cut20:30:173
1222499117,3cyclictest13432-21cat20:19:593
1222499115,4cyclictest39-21ksoftirqd/321:10:033
1222499115,4cyclictest39-21ksoftirqd/321:10:033
1222499110,10cyclictest31775-21sort20:55:203
1222299119,1cyclictest30577-21wc19:45:262
1222299119,1cyclictest26714-21grep23:00:212
1222299119,1cyclictest13301-21grep22:30:262
1222299119,1cyclictest12499-21grep20:15:242
1222299119,1cyclictest1104-21wc22:05:262
1222299118,2cyclictest6788-21tr20:05:162
1222299118,2cyclictest462-21grep00:20:182
1222299118,2cyclictest27918-21grep21:55:172
1222299118,2cyclictest27026-21tr19:40:182
1222299118,2cyclictest24693-21wc20:40:192
1222299118,2cyclictest23786-21taskset19:30:432
1222299118,2cyclictest2266-21grep23:15:252
1222299118,2cyclictest10250-21tr21:20:002
1222299115,5cyclictest2585-21kworker/u8:123:30:172
1222299114,4cyclictest32-21ksoftirqd/221:35:142
1222299111,7cyclictest29075-21gltestperf23:05:172
1222299110,11cyclictest0-21swapper/219:15:142
1221899119,1cyclictest26081-21grep19:35:251
1221899119,1cyclictest18878-21grep19:20:251
1221899118,2cyclictest32731-21grep22:05:181
1221899118,2cyclictest30863-21sed19:45:271
1221899118,2cyclictest26493-21grep21:50:281
1221899118,2cyclictest25700-21cron23:00:011
1221899118,2cyclictest241rcuc/120:00:141
1221899118,2cyclictest22731-21tr21:45:171
1221899118,2cyclictest18840-21seq22:45:151
1221899117,3cyclictest7802-21chrt23:30:031
1221899115,1cyclictest25-21ksoftirqd/123:55:001
1221899111,7cyclictest141rcu_preempt21:42:411
12218991110,1cyclictest0-21swapper/100:00:171
1221899110,1cyclictest141rcu_preempt20:55:201
1221599119,1cyclictest17353-21cron21:35:000
1221599118,2cyclictest6577-21cut23:25:200
1221599118,2cyclictest4765-21grep20:00:170
1221599118,2cyclictest3793-21chrt19:59:230
1221599118,2cyclictest32045-21wc19:50:140
1221599118,2cyclictest28072-21taskset23:04:440
1221599118,2cyclictest26531-21chrt00:05:370
1221599118,2cyclictest26531-21chrt00:05:360
1221599118,2cyclictest26499-21sh19:40:000
1221599118,2cyclictest25800-21cat20:45:000
1221599118,2cyclictest21957-21grep22:50:140
1221599118,2cyclictest2191-21tr22:10:160
1221599118,2cyclictest18785-21cat20:30:000
1221599118,2cyclictest16497-21tr22:40:140
1221599118,2cyclictest15905-21taskset19:15:220
1221599118,2cyclictest15336-21expr23:45:170
1221599118,2cyclictest15250-21tr20:20:260
1221599118,2cyclictest12851-21ls22:30:210
1221599118,2cyclictest11840-21grep23:35:260
1221599113,3cyclictest8106-21apt-get21:15:130
1221599112,2cyclictest6427-21munin-run22:19:590
1221599110,10cyclictest20554-21sed22:45:280
1222499108,1cyclictest85892sleep322:22:293
1222499108,1cyclictest6146-21cut23:25:163
1222499108,1cyclictest31392-21sed00:15:263
1222499108,1cyclictest29159-21apache_processe19:45:123
1222499108,1cyclictest22360-21chrt00:00:033
1222499108,1cyclictest14394-21sed20:20:193
1222499107,2cyclictest32315-21wc23:10:273
1222499107,2cyclictest15273-21tr23:45:173
1222499105,2cyclictest39-21ksoftirqd/322:30:163
1222499103,2cyclictest0-21swapper/322:55:193
1222499101,8cyclictest0-21swapper/320:10:143
1222499100,9cyclictest15973-21grep19:15:233
1222299108,1cyclictest26922sleep221:02:522
1222299108,1cyclictest14483-21cat22:35:152
1222299107,2cyclictest4165-21basename20:00:142
1222299107,2cyclictest25568-21grep21:50:192
1222299107,2cyclictest24176-21taskset00:00:502
1222299107,2cyclictest18436-21grep19:20:202
1222299107,2cyclictest16673-21tr22:40:152
1222299107,2cyclictest16472-21head21:30:212
1222299107,2cyclictest13841-21tr23:40:222
1222299106,3cyclictest7385-21sed21:10:282
1222299106,3cyclictest7385-21sed21:10:272
1222299104,5cyclictest32-21ksoftirqd/223:10:162
1222299102,4cyclictest225-21systemd-journal23:22:132
1222299100,4cyclictest225-21systemd-journal00:09:252
1222299100,4cyclictest225-21systemd-journal00:09:242
1221899108,2cyclictest3594-21expr21:05:181
1221899108,2cyclictest18660-21cat21:35:211
1221899108,1cyclictest7856-21basename21:15:011
1221899108,1cyclictest7856-21basename21:15:001
1221899108,1cyclictest27066-21grep19:40:181
1221899108,1cyclictest241rcuc/121:20:121
1221899108,1cyclictest11092-21chrt22:29:091
1221899107,2cyclictest725-21cat00:20:201
1221899107,2cyclictest25366-21sed20:40:251
1221899107,2cyclictest21505-21cut22:50:121
1221899102,3cyclictest225-21systemd-journal20:38:311
1221899101,7cyclictest4213-21cat22:15:001
1221899101,1cyclictest141rcu_preempt23:35:151
1221899101,1cyclictest141rcu_preempt22:20:001
1221899100,9cyclictest13361-21uname19:10:181
1221899100,9cyclictest13361-21uname19:10:181
1221899100,2cyclictest225-21systemd-journal23:30:011
1221599108,2cyclictest24080-21unixbench_multi21:45:280
1221599108,1cyclictest9860-21which22:25:170
1221599108,1cyclictest6665-21grep20:05:140
1221599108,1cyclictest19038-21chrt19:21:280
1221599108,1cyclictest15964-21chrt22:37:530
1221599108,1cyclictest13088-21cat21:25:170
1221599108,1cyclictest12697-21aten2_r7power_p23:40:160
1221599107,2cyclictest9233-21cat00:40:000
1221599107,2cyclictest4949-21taskset23:21:520
1221599107,2cyclictest31910-21sh00:20:000
1221599107,2cyclictest30421-21cat22:00:180
1221599107,2cyclictest29219-21cut19:45:150
1221599107,2cyclictest28371-21taskset20:50:070
1221599107,2cyclictest2414-21expr00:25:160
1221599107,2cyclictest21989-21chrt19:30:100
1221599107,2cyclictest20413-21head19:25:160
1221599107,2cyclictest20368-21threads20:30:240
1221599100,9cyclictest51582sleep021:09:030
122249996,2cyclictest28154-21grep00:10:203
122249991,4cyclictest225-21systemd-journal20:40:003
122229996,2cyclictest30920-21grep20:55:152
122189998,1cyclictest26518-21seq00:05:291
122189998,1cyclictest26518-21seq00:05:281
122189998,1cyclictest11930-21expr22:30:141
122189997,2cyclictest6899-21expr20:05:151
122189997,2cyclictest16674-21cat21:30:251
122189997,1cyclictest948-21grep23:15:141
122189997,1cyclictest8601-21grep00:35:301
122189993,3cyclictest0-21swapper/119:55:161
122159997,1cyclictest106662sleep020:10:530
122189987,1cyclictest14092-21grep23:40:261
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional