You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-01 - 08:42

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack5slot4s.osadl.org (updated Sun Jun 29, 2025 12:44:41)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3046899186182,1cyclictest741-21snmpd07:13:321
3046399180171,8cyclictest0-21swapper/007:10:000
3047299127125,1cyclictest0-21swapper/207:10:012
30240212398,10sleep10-21swapper/107:09:191
300822121108,10sleep20-21swapper/207:07:152
251432119106,9sleep30-21swapper/307:05:043
30162211198,10sleep00-21swapper/007:08:170
30468996863,4cyclictest22975-21kworker/u16:3+events_unbound10:40:021
30468996358,4cyclictest31357-21kworker/u16:0+events_unbound12:35:021
49082510,1sleep04912-21kthreadcore10:50:220
30468995147,3cyclictest22975-21kworker/u16:3+events_unbound09:00:261
30463995145,4cyclictest22975-21kworker/u16:3+events_unbound10:30:290
30468994945,3cyclictest22975-21kworker/u16:3+events_unbound07:15:311
30468994844,3cyclictest21338-21kworker/u16:2+events_unbound10:35:221
30468994834,9cyclictest21338-21kworker/u16:2+events_unbound12:05:181
60032450,2chrt6008-21irqcore10:15:201
30468994541,3cyclictest21338-21kworker/u16:2+events_unbound07:25:311
30468994338,4cyclictest21338-21kworker/u16:2+events_unbound08:55:291
30468994238,3cyclictest22975-21kworker/u16:3+events_unbound10:25:091
30468994238,3cyclictest22975-21kworker/u16:3+events_unbound09:45:271
30468994238,3cyclictest21338-21kworker/u16:2+events_unbound07:30:281
30468994138,2cyclictest31357-21kworker/u16:0+events_unbound11:35:181
30468994137,3cyclictest21338-21kworker/u16:2+events_unbound08:25:291
30468994135,3cyclictest22975-21kworker/u16:3+events_unbound09:30:301
30468994036,3cyclictest21338-21kworker/u16:2+events_unbound09:50:281
30468993935,3cyclictest21338-21kworker/u16:2+events_unbound12:25:201
30468993934,4cyclictest31357-21kworker/u16:0+events_unbound12:15:081
30468993933,4cyclictest22975-21kworker/u16:3+events_unbound08:10:201
30468993632,3cyclictest22975-21kworker/u16:3+events_unbound09:20:241
30468993632,3cyclictest21338-21kworker/u16:2+events_unbound07:55:181
30463993631,4cyclictest21338-21kworker/u16:2+events_unbound07:35:020
30468993532,2cyclictest22975-21kworker/u16:3+events_unbound09:25:011
30468993430,3cyclictest22975-21kworker/u16:3+events_unbound08:15:161
30468993430,3cyclictest21338-21kworker/u16:2+events_unbound11:40:191
30468993330,2cyclictest31357-21kworker/u16:0+events_unbound12:10:001
30468993330,2cyclictest31357-21kworker/u16:0+events_unbound12:00:131
30468993229,2cyclictest21338-21kworker/u16:2+events_unbound09:40:391
30468993129,2cyclictest21338-21kworker/u16:2+events_unbound11:50:211
30468993126,4cyclictest21338-21kworker/u16:2+events_unbound11:00:081
30468993026,3cyclictest21338-21kworker/u16:2+events_unbound08:20:221
30468993026,3cyclictest21338-21kworker/u16:2+events_unbound07:40:281
30468993026,2cyclictest21338-21kworker/u16:2+events_unbound09:05:151
30468993025,4cyclictest21338-21kworker/u16:2+events_unbound08:35:151
30468993018,9cyclictest7078-21ntp_states08:40:241
30468993018,10cyclictest29034-21diskstats10:45:121
30468992926,2cyclictest31357-21kworker/u16:0+events_unbound11:05:011
30468992921,4cyclictest22975-21kworker/u16:3+events_unbound10:30:221
30468992920,8cyclictest8390-21munin-run09:15:001
30468992824,3cyclictest21338-21kworker/u16:2+events_unbound10:50:251
30468992717,8cyclictest21338-21kworker/u16:2+events_unbound07:39:201
30468992623,2cyclictest21338-21kworker/u16:2+events_unbound10:20:021
30468992623,2cyclictest21338-21kworker/u16:2+events_unbound10:00:141
30468992623,2cyclictest21338-21kworker/u16:2+events_unbound10:00:141
30468992623,2cyclictest21338-21kworker/u16:2+events_unbound08:00:171
30468992622,3cyclictest22975-21kworker/u16:3+events_unbound08:30:161
30468992521,3cyclictest31357-21kworker/u16:0+events_unbound11:55:011
30468992521,3cyclictest31357-21kworker/u16:0+events_unbound11:15:011
30468992421,2cyclictest22975-21kworker/u16:3+events_unbound07:20:191
30468992421,2cyclictest21338-21kworker/u16:2+events_unbound09:35:011
30468992420,3cyclictest22975-21kworker/u16:3+events_unbound07:45:361
30468992320,3cyclictest21338-21kworker/u16:2+events_unbound08:45:001
30468992320,2cyclictest31357-21kworker/u16:0+events_unbound11:25:201
30468992319,3cyclictest22975-21kworker/u16:3+events_unbound10:10:021
30468992312,8cyclictest9060-21diskstats09:15:161
30463992319,3cyclictest21338-21kworker/u16:2+events_unbound10:45:090
30463992218,3cyclictest22975-21kworker/u16:3+events_unbound09:20:320
30468992117,3cyclictest22975-21kworker/u16:3+events_unbound08:51:361
30468992111,8cyclictest31357-21kworker/u16:0+events_unbound11:34:001
30463992117,3cyclictest22975-21kworker/u16:3+events_unbound07:15:040
30463992117,3cyclictest0-21swapper/009:35:000
30468992018,2cyclictest31357-21kworker/u16:0+events_unbound10:55:011
30463992017,2cyclictest0-21swapper/010:40:000
30463992016,3cyclictest21338-21kworker/u16:2+events_unbound11:20:230
30472991914,4cyclictest21338-21kworker/u16:2+events_unbound08:15:022
3046899198,6cyclictest0-21swapper/112:23:201
3046899198,10cyclictest741-21snmpd10:06:401
30468991916,2cyclictest22975-21kworker/u16:3+events_unbound08:05:451
30468991915,3cyclictest31357-21kworker/u16:0+events_unbound11:20:101
30463991915,3cyclictest7670-21kworker/u16:1+events_unbound09:15:270
30463991915,3cyclictest7670-21kworker/u16:1+events_unbound09:10:300
30468991815,2cyclictest31357-21kworker/u16:0+events_unbound12:30:011
30468991814,3cyclictest21338-21kworker/u16:2+events_unbound11:48:361
3046899170,16cyclictest0-21swapper/109:55:141
3046899170,16cyclictest0-21swapper/109:55:141
3046399178,8cyclictest0-21swapper/011:00:000
3046399171,8cyclictest13711-21/usr/sbin/munin08:50:260
30463991713,3cyclictest31357-21kworker/u16:0+events_unbound11:40:230
3046399170,16cyclictest0-21swapper/009:55:140
3046399170,16cyclictest0-21swapper/009:55:140
30472991613,2cyclictest0-21swapper/208:45:002
30468991614,2cyclictest31357-21kworker/u16:0+events_unbound11:11:201
3046399165,8cyclictest0-21swapper/012:28:580
30463991611,4cyclictest22975-21kworker/u16:3+events_unbound09:00:200
235812160,1sleep023587-21cut07:30:210
30472991511,3cyclictest22975-21kworker/u16:3+events_unbound07:15:022
3046899159,3cyclictest22975-21kworker/u16:3+flush-8:007:50:581
3046899159,3cyclictest22975-21kworker/u16:3+flush-8:007:50:581
3046399159,5cyclictest21338-21kworker/u16:2+events_unbound07:40:190
3046399156,6cyclictest7850irq/24-ahci[0000:00:1f.2]08:00:010
3046399154,7cyclictest0-21swapper/012:30:590
3046399154,6cyclictest24243-21apt-get08:30:130
3046399152,12cyclictest21532-21kworker/0:1+events08:15:140
3046399152,11cyclictest16024-21turbostat.cron11:55:000
30463991510,4cyclictest21338-21kworker/u16:2+events_unbound08:20:270
30463991510,3cyclictest16849-21cat12:00:000
3046399150,10cyclictest0-21swapper/009:50:200
30472991410,3cyclictest9035-21kthreadcore11:00:152
3046399147,5cyclictest0-21swapper/007:27:420
3046399144,8cyclictest0-21swapper/010:23:580
3046399144,5cyclictest0-21swapper/008:05:250
3046399141,8cyclictest16237-21taskset10:28:340
30463991410,3cyclictest29958-21cpuspeed_turbos08:35:160
3046399140,9cyclictest19310-21latency_hist11:15:000
3046399140,1cyclictest0-21swapper/012:05:140
3046399140,13cyclictest16930-21aten2_r5power_c12:00:120
3047299139,2cyclictest25417-21latency_hist10:00:002
3047299139,2cyclictest25417-21latency_hist10:00:002
3047299131,4cyclictest49122kthreadcore10:50:222
30472991310,2cyclictest0-21swapper/207:35:002
3046399139,3cyclictest23698-21tr08:00:150
3046399139,3cyclictest23107-21irqrtprio10:35:160
3046399137,4cyclictest21532-21kworker/0:1+events08:45:260
3046399133,9cyclictest21532-21kworker/0:1+events11:25:140
3046399133,8cyclictest0-21swapper/008:13:220
3046399133,6cyclictest31121-21turbostat10:05:000
3046399133,6cyclictest31121-21turbostat10:05:000
3046399133,5cyclictest0-21swapper/008:40:190
3046399133,5cyclictest0-21swapper/008:25:190
3046399132,6cyclictest13613-21irqstats07:50:190
3046399132,6cyclictest13613-21irqstats07:50:180
3046399131,10cyclictest10251-21munin-run10:20:000
30463991310,2cyclictest32083-21aten2_r5power_p10:10:150
3046399130,8cyclictest12598-21kthreadcore07:20:210
3046399130,5cyclictest765-21runrttasks12:15:000
3046399130,1cyclictest0-21swapper/012:15:200
3047299127,3cyclictest4673-21tail12:25:152
3047299127,3cyclictest11489-21sed11:50:112
3047299120,3cyclictest2906-21grep09:35:312
3047299120,2cyclictest27478-21apt-get07:35:022
3046399128,3cyclictest1984-21ntp_offset09:05:230
3046399128,2cyclictest0-21swapper/011:40:000
3046399126,4cyclictest19821-21cpuspeed09:25:160
3046399125,5cyclictest0-21swapper/012:36:500
3046399125,5cyclictest0-21swapper/009:49:060
3046399125,2cyclictest0-21swapper/011:20:000
3046399125,2cyclictest0-21swapper/009:35:270
3046399122,9cyclictest21532-21kworker/0:1+events09:40:220
3046399122,8cyclictest0-21swapper/007:48:040
3046399122,5cyclictest13078-21idleruntime-cro11:05:000
3046399120,11cyclictest23524-21ps08:55:270
3047299118,2cyclictest0-21swapper/207:40:182
3047299118,2cyclictest0-21swapper/207:25:002
3047299117,2cyclictest6624-21cut09:10:242
3047299116,3cyclictest8950-21tail08:45:202
3047299114,5cyclictest18748-21grep11:10:222
3047299112,7cyclictest741-21snmpd10:30:112
3047299110,10cyclictest7535-21ps10:15:212
3046399114,5cyclictest0-21swapper/011:08:060
3046399113,5cyclictest0-21swapper/011:33:090
3046399112,8cyclictest0-21swapper/011:45:370
3046399111,8cyclictest0-21swapper/012:20:170
3046399111,8cyclictest0-21swapper/010:06:430
3047299108,1cyclictest0-21swapper/210:35:242
3047299107,2cyclictest7591-21sendmail11:00:002
3047299105,4cyclictest322-21systemd-journal11:06:352
3047299105,4cyclictest322-21systemd-journal10:05:002
3047299105,4cyclictest322-21systemd-journal10:05:002
3047299105,4cyclictest25579-21sed09:00:192
3047299104,5cyclictest12470-21apt-get07:50:122
3047299104,5cyclictest12470-21apt-get07:50:122
3047299104,4cyclictest5833-21cut08:10:212
3047299104,4cyclictest31786-21cpu11:35:152
3047299104,4cyclictest22443-21cut11:20:172
3047299102,3cyclictest23842-21sh08:00:162
3047299100,9cyclictest16398-21chrt11:56:532
3047299100,8cyclictest952-21grep10:45:232
3047299100,8cyclictest4314-21sed09:40:182
3047299100,8cyclictest32222-21kthreadcore08:35:222
3047299100,8cyclictest25281-21latency_hist11:25:002
3047299100,8cyclictest23924-21grep08:55:302
3047299100,8cyclictest22663-21tr12:05:152
3047299100,8cyclictest21312-21awk12:00:232
3047299100,8cyclictest2102-21cut10:10:222
3047299100,8cyclictest19366-21fschecks_time08:25:172
3047299100,8cyclictest16890-21aten2_r5power_v07:25:172
3047299100,8cyclictest13887-21munin-plugin-st12:40:002
3047299100,8cyclictest12259-21sed12:30:272
3047299100,8cyclictest11222-21cut09:15:212
3047299100,8cyclictest10853-21cut07:45:262
3047299100,5cyclictest21991-21cut09:25:232
3047299100,5cyclictest14318-21cut08:20:212
3047299100,3cyclictest11401-21tr10:25:102
3047299100,1cyclictest0-21swapper/208:05:232
3047299100,10cyclictest0-21swapper/212:15:112
3047299100,10cyclictest0-21swapper/210:40:032
304729998,1cyclictest0-21swapper/207:55:172
304729996,3cyclictest351ktimers/209:50:132
304729994,4cyclictest765-21runrttasks12:23:502
304729994,4cyclictest600-21dbus-daemon11:35:002
304729991,7cyclictest741-21snmpd10:06:012
304729991,6cyclictest10768-21latency_hist11:50:002
304729990,8cyclictest5012-21tr11:40:112
304729990,8cyclictest26650-21cut08:30:222
304729990,8cyclictest14386-21tr09:20:202
304729990,8cyclictest11968-21cut09:45:222
304729990,4cyclictest30242-21wc09:05:142
304729990,4cyclictest28195-21chrt12:14:452
304729990,4cyclictest25366-21cut09:30:182
304729990,4cyclictest19487-21chrt11:16:092
304729990,4cyclictest13826-21aten2_r5power_v08:50:152
304729990,4cyclictest10853-21chrt10:24:352
1262281,4sleep30-21swapper/312:20:003
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
none15:53:397
7647270,2sleep30-21swapper/308:40:353
28101270,5sleep30-21swapper/307:35:023
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
none15:53:396
8251260,4sleep30-21swapper/308:45:123
7852260,4sleep30-21swapper/311:00:083
7667260,4sleep30-21swapper/308:15:023
5041260,4sleep30-21swapper/310:15:153
31994260,4sleep30-21swapper/310:10:143
30427263,2sleep30-21swapper/311:26:013
30118260,4sleep30-21swapper/309:05:133
29703260,4sleep30-21swapper/308:35:143
28245260,5sleep30-21swapper/310:43:413
24741260,5sleep30-21swapper/309:28:533
22385260,4sleep30-21swapper/312:05:133
22261260,4sleep30-21swapper/307:30:133
16315263,2sleep30-21swapper/311:55:513
13778260,2sleep30-21swapper/309:18:593
13710260,4sleep30-21swapper/308:50:143
13430260,2sleep30-21swapper/311:06:373
12055260,2sleep30-21swapper/307:48:013
10332260,6sleep30-21swapper/311:47:133
none15:53:395
none15:53:395
none15:53:395
none15:53:395
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional