You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-23 - 03:50

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot7s.osadl.org (updated Tue Dec 23, 2025 00:43:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
572329582,9sleep00-21swapper/019:06:170
387229072,14sleep10-21swapper/119:05:021
599928566,14sleep20-21swapper/219:09:542
589828168,9sleep30-21swapper/319:08:333
610699224,14cyclictest8622-21apt-get23:45:033
610699201,1cyclictest14544-21latency_hist22:50:003
6100991814,2cyclictest29941-21unixbench_singl00:25:271
610399170,0cyclictest0-21swapper/221:40:112
610099169,6cyclictest3516-21wc22:25:201
610099168,3cyclictest22757-21cron21:59:591
610099151,13cyclictest12266-21aten2_r7power_v20:30:141
609899155,5cyclictest16819-21perf22:55:000
6098991512,2cyclictest13267-21apt19:25:010
6098991512,2cyclictest13267-21apt19:25:000
6106991411,2cyclictest25199-21turbostat.cron19:50:003
610399142,11cyclictest28742-21cron21:05:012
6103991410,3cyclictest9381-21cron22:40:002
6103991410,3cyclictest5775-21cron23:40:002
6103991410,3cyclictest24184-21grep00:15:182
6103991410,3cyclictest20508-31su00:07:572
610099148,5cyclictest24011-21wc22:00:211
6100991411,2cyclictest1672-21wc20:05:271
610099140,8cyclictest19006-21kworker/1:223:10:151
609899142,6cyclictest25184-21perf22:04:590
6098991410,3cyclictest7945-21grep21:25:270
6098991410,3cyclictest2404-21wc20:10:000
6106991310,2cyclictest8095-21grep20:20:173
6106991310,2cyclictest23797-21wc20:55:003
6106991310,2cyclictest21243-21tr00:10:153
6106991310,2cyclictest18175-21cron00:04:593
6106991310,2cyclictest16060-21wc20:35:253
610399139,3cyclictest11443-21basename23:50:162
610399139,2cyclictest13353-21wc22:45:252
610399134,4cyclictest225-21systemd-journal23:59:552
6103991310,2cyclictest9432-21head23:45:162
6103991310,2cyclictest941-21sed20:05:182
6103991310,2cyclictest4819-21wc22:30:002
6103991310,2cyclictest29918-21awk22:15:002
6103991310,2cyclictest27539-21tail00:20:292
6103991310,2cyclictest19188-21munin-run23:00:002
6103991310,2cyclictest15248-21grep19:25:262
610399130,12cyclictest26342-21cron21:00:002
610399130,12cyclictest25899-21grep23:10:282
610399130,12cyclictest12383-21sed19:20:232
610399130,12cyclictest12383-21sed19:20:232
610399130,12cyclictest0-21swapper/220:20:142
610399130,10cyclictest0-21swapper/220:50:152
610099137,5cyclictest20520-21chrt00:07:571
610099137,5cyclictest12393-21cut22:45:181
6100991310,2cyclictest9946-21tr22:40:151
6100991310,2cyclictest29263-21cut21:05:131
6100991310,2cyclictest2649-21cut21:15:201
6100991310,2cyclictest24049-21wc19:45:171
6100991310,2cyclictest21948-21tr19:40:201
610099130,12cyclictest2329-21taskset00:35:381
610099130,12cyclictest18716-21grep22:55:281
609899136,6cyclictest23775-21apt-get20:55:000
6098991310,2cyclictest9378-21cron22:40:010
6098991310,2cyclictest8205-21basename23:45:000
6098991310,2cyclictest5436-21cat21:20:250
6098991310,2cyclictest27444-21cat00:20:280
6098991310,2cyclictest19835-21cat19:35:270
6098991310,2cyclictest17800-21head22:55:180
6098991310,2cyclictest1637-21cat22:20:290
6098991310,2cyclictest15118-21grep20:35:160
6098991310,2cyclictest14417-21cron22:50:000
609899130,1cyclictest0-21swapper/020:25:120
609899130,12cyclictest27359-21grep21:00:170
609899130,0cyclictest0-21swapper/019:10:130
610699129,2cyclictest9448-21head19:15:163
610699129,2cyclictest6578-21grep19:10:153
610699129,2cyclictest566-21expr22:20:173
610699129,2cyclictest29971-21grep21:05:163
610699129,2cyclictest29386-21tr00:25:213
610699129,2cyclictest2876-21find21:15:223
610699129,2cyclictest26518-21cat21:00:003
610699129,2cyclictest21912-21wc21:55:223
610699129,2cyclictest17675-21grep20:40:173
610699129,2cyclictest13259-21cron19:25:003
610699129,2cyclictest13259-21cron19:25:003
6106991210,1cyclictest25707-21grep19:50:153
610399129,2cyclictest7747-21tail23:40:282
610399129,2cyclictest7044-21cron20:20:002
610399129,2cyclictest6579-21grep22:30:262
610399129,2cyclictest4665-21grep21:20:162
610399129,2cyclictest32696-21cut21:10:192
610399129,2cyclictest32-21ksoftirqd/220:40:132
610399129,2cyclictest25674-21grep22:05:162
610399129,2cyclictest20994-21head20:45:262
610399129,2cyclictest19389-21grep21:50:222
610399129,2cyclictest18563-21tr19:35:162
610399129,2cyclictest1830-21tr21:15:152
610399129,2cyclictest12826-21grep21:35:262
610399129,1cyclictest345-21tr22:20:172
610399128,3cyclictest2675-21cron00:40:002
610399128,3cyclictest16619-21cron20:40:002
610399126,5cyclictest477-21rs:main2
610399126,4cyclictest14973-21munin-node22:50:142
610399121,4cyclictest32-21ksoftirqd/219:55:282
6103991210,1cyclictest32-21ksoftirqd/219:40:252
6103991210,1cyclictest21675-21wc23:05:002
610099129,2cyclictest9258-21grep19:15:151
610099129,2cyclictest5525-21sort22:30:171
610099129,2cyclictest5359-21grep23:35:261
610099129,2cyclictest31997-21cut23:25:161
610099129,2cyclictest31942-21wc00:30:231
610099129,2cyclictest31390-21grep22:15:211
610099129,2cyclictest27175-21grep19:50:271
610099129,2cyclictest19938-21grep21:50:281
610099129,2cyclictest1853-21grep22:20:311
610099129,2cyclictest15245-21tail20:35:181
610099128,3cyclictest28609-21taskset21:03:231
610099128,3cyclictest25464-21sed20:55:231
610099128,3cyclictest20048-21sed20:45:171
610099128,3cyclictest13247-21sh21:40:001
610099127,4cyclictest7180-21wc20:20:001
610099121,6cyclictest225-21systemd-journal19:33:111
610099120,11cyclictest0-21swapper/119:20:291
610099120,11cyclictest0-21swapper/119:20:281
609899129,2cyclictest4629-21sh22:30:000
609899129,2cyclictest3054-21df20:10:170
609899129,2cyclictest30074-21grep20:00:020
609899129,2cyclictest28277-21grep23:15:290
609899129,2cyclictest23871-21grep19:45:160
609899129,2cyclictest21155-21grep23:00:270
609899129,2cyclictest2079-21head23:30:180
609899129,2cyclictest18712-21tr00:05:150
609899129,2cyclictest15184-21sed21:40:240
609899129,2cyclictest12925-21head20:30:170
609899129,2cyclictest10839-21sleep19:20:000
609899129,1cyclictest0-21swapper/021:15:150
610699119,2cyclictest8180-21cron23:45:003
610699119,2cyclictest27423-21grep21:00:173
610699119,1cyclictest27803-21grep23:15:243
610699118,2cyclictest9934-21tr22:40:153
610699118,2cyclictest6968-21taskset20:19:033
610699118,2cyclictest5426-21cat21:20:243
610699118,2cyclictest3592-21cat20:10:213
610699118,2cyclictest32293-21run-parts22:20:003
610699118,2cyclictest28295-21expr19:55:173
610699118,2cyclictest25741-21wc22:05:153
610699118,2cyclictest1344-21grep00:35:213
610699110,10cyclictest20114-21taskset21:51:333
610399118,2cyclictest9809-21cut20:25:132
610399118,2cyclictest7111-21sed21:25:192
610399118,2cyclictest32114-21chrt22:18:002
610399118,2cyclictest31229-21chrt21:08:482
610399118,2cyclictest27562-21cron19:55:002
610399118,2cyclictest21870-21grep23:05:152
610399118,2cyclictest16682-21uniq00:00:192
610399118,2cyclictest16651-21grep21:45:172
610399118,2cyclictest15871-21taskset19:30:042
610399117,2cyclictest10824-21munin-run19:20:002
610399110,9cyclictest0-21swapper/221:55:132
610099118,2cyclictest9778-21grep23:45:191
610099118,2cyclictest8570-21grep20:20:211
610099118,2cyclictest3273-21tail20:10:181
610099118,2cyclictest29504-21sed22:10:261
610099118,2cyclictest28579-21grep19:55:201
610099118,2cyclictest26591-21chrt23:15:161
610099118,2cyclictest26422-21grep00:20:181
610099118,2cyclictest22774-21tail00:10:261
610099118,2cyclictest22411-21sed20:50:191
610099118,2cyclictest2070-21cut23:30:181
610099118,2cyclictest20169-21grep23:00:191
610099118,2cyclictest14825-21tr23:55:221
610099118,2cyclictest10929-21wc21:35:001
609899118,2cyclictest831-21tr00:35:190
609899118,2cyclictest7741-21expr20:20:140
609899118,2cyclictest6697-21taskset20:15:360
609899118,2cyclictest31848-21tr21:10:140
609899118,2cyclictest29471-21grep00:25:230
609899118,2cyclictest26824-21sendmail_mailtr19:50:230
609899118,2cyclictest25340-21awk23:10:220
609899118,2cyclictest22189-21uname00:10:200
609899118,2cyclictest17791-21chrt21:47:120
609899118,2cyclictest13998-21cat19:25:140
609899118,2cyclictest10888-21cat21:35:000
609899116,5cyclictest1436-21sendmail-mta20:47:560
609899114,6cyclictest32359-21awk20:04:590
609899110,10cyclictest21081-21expr21:55:160
610699108,2cyclictest5748-21chrt23:39:423
610699108,2cyclictest23655-21taskset23:06:303
610699108,1cyclictest8110-21cat22:35:163
610699108,1cyclictest31838-21cut21:10:143
610699108,1cyclictest29252-21tr23:20:163
610699108,1cyclictest24936-21head23:10:193
610699108,1cyclictest20970-21grep19:40:133
610699108,1cyclictest20378-21taskset00:07:503
610699108,1cyclictest18324-21grep22:55:243
610699108,1cyclictest182962sleep319:35:133
610699108,1cyclictest155872sleep319:29:253
610699108,1cyclictest15095-21expr22:50:153
610699108,1cyclictest14615-21cut23:55:213
610699108,1cyclictest139832sleep320:31:433
610699108,1cyclictest13563-21basename21:40:123
610699108,1cyclictest11868-21cron20:30:003
610699108,1cyclictest106952sleep321:33:333
610699107,2cyclictest939-21taskset23:29:043
610699107,2cyclictest8043-21sed21:25:273
610699107,2cyclictest6570-21grep22:30:273
610699107,2cyclictest28581-21head22:10:173
610699107,2cyclictest26156-21grep00:20:163
610699107,2cyclictest16171-21expr19:30:153
610699103,3cyclictest225-21systemd-journal20:04:593
610699101,5cyclictest225-21systemd-journal00:35:003
610399108,1cyclictest62892sleep219:10:042
610399108,1cyclictest28243-21apache_accesses00:25:132
610399108,1cyclictest27789-21cut23:15:242
610399108,1cyclictest22849-21taskset00:10:312
610399108,1cyclictest10193-21wc21:30:252
610399107,2cyclictest9894-21tr22:40:152
610399107,2cyclictest23822-21head19:45:152
610399107,2cyclictest23123-21wget22:00:162
610399107,2cyclictest2090-21sed23:30:182
610399107,2cyclictest13399-21grep20:30:212
610099108,1cyclictest90352sleep122:35:381
610099108,1cyclictest6125-21awk23:40:171
610099108,1cyclictest31840-21grep20:00:271
610099108,1cyclictest17584-21cut20:40:171
610099108,1cyclictest16580-21tune2fs21:45:171
610099108,1cyclictest11622sleep121:13:321
610099108,1cyclictest10076-21expr20:25:141
610099107,2cyclictest6823-21grep19:10:151
610099107,2cyclictest25519-21chrt00:19:191
610099107,2cyclictest23535-21head23:05:281
610099107,2cyclictest16321-21grep00:00:171
610099107,2cyclictest14859-21grep21:40:211
610099107,2cyclictest13833-21wc19:25:141
610099107,2cyclictest13031-21sed23:50:271
610099101,4cyclictest26177-21gltestperf22:05:181
610099100,7cyclictest14776-21aten2_r7power_p22:50:151
609899108,1cyclictest4026-21cat23:35:150
609899108,1cyclictest31349-21apache_volume23:25:110
609899108,1cyclictest29498-21grep21:05:140
609899108,1cyclictest21470-21expr19:40:150
609899108,1cyclictest17952-21chrt19:33:460
609899108,1cyclictest13067-21chrt23:50:370
609899108,1cyclictest12630-21cat21:35:250
609899107,2cyclictest9660-21cat22:40:120
609899107,2cyclictest5361-21expr22:30:150
609899107,2cyclictest30203-21hostname22:15:130
609899107,2cyclictest29331-21expr23:20:170
609899107,2cyclictest28011-21grep22:10:140
609899107,2cyclictest26027-21grep22:05:170
609899107,2cyclictest24535-21grep20:55:160
609899107,2cyclictest24256-21tail00:15:180
609899107,2cyclictest22434-21grep23:05:180
609899107,2cyclictest19606-21find21:50:240
609899107,2cyclictest17231-21cat20:40:150
609899107,2cyclictest16369-21grep00:00:170
609899107,2cyclictest13916-21tr23:55:170
609899102,4cyclictest225-21systemd-journal00:35:000
61069998,1cyclictest23104-21cut22:00:153
61069998,1cyclictest19518-21cut20:45:153
61069998,1cyclictest11580-21cut23:50:173
61069998,1cyclictest1148-21sort20:05:213
61069997,1cyclictest1707-21expr23:30:163
61069996,2cyclictest19725-21grep23:00:153
61069996,2cyclictest16590-21cut21:45:173
61069991,3cyclictest0-21swapper/300:15:153
61069990,9cyclictest2937-21expr22:25:163
61039997,1cyclictest30777-21chrt20:00:162
61039997,1cyclictest289322sleep223:20:022
61039996,2cyclictest31632-21tr23:25:152
61009997,1cyclictest4494-21sed21:20:151
61009996,2cyclictest7329-21wc21:25:201
61009996,2cyclictest29468-21df23:20:171
61009996,2cyclictest18955-21sort19:35:181
60989997,1cyclictest8491-21apt-config23:45:010
61069987,1cyclictest12706-21timerwakeupswit21:35:253
61039986,1cyclictest31257-21cut00:30:172
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional