You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-09-16 - 22:20

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Tue Sep 16, 2025 12:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
9617992721,5cyclictest0-21swapper/312:36:233
9605992621,4cyclictest7664-21apt-get12:36:220
961099250,24cyclictest17187-21latency_hist10:45:001
9605992520,4cyclictest28911-21munin-run07:50:000
9617992423,1cyclictest23017-21apt-get09:51:223
9617992422,1cyclictest0-21swapper/308:59:593
9605992422,1cyclictest30201-21sendmail-msp09:00:000
9617992322,1cyclictest13734-21apt-get08:26:193
9617992316,1cyclictest8556-21cat08:15:003
961799230,22cyclictest1710-21apt-get08:01:173
9613992322,0cyclictest11340-21apt-get08:21:222
961399230,22cyclictest4117-21apt-get08:06:222
961399230,22cyclictest2830-21sendmail-msp11:20:002
9610992321,1cyclictest0-21swapper/107:21:211
961099230,2cyclictest27585-21awk11:05:001
961099230,1cyclictest583-21avahi-daemon12:20:011
961099230,1cyclictest26512-21munin-run07:45:001
9605992323,0cyclictest0-21swapper/009:51:220
9605992319,3cyclictest16133-21apt-get08:31:210
9617992220,1cyclictest593-21dbus-daemon10:24:593
9617992220,1cyclictest10423-21cron11:35:013
9617992219,3cyclictest0-21swapper/308:31:223
9617992219,3cyclictest0-21swapper/307:50:003
961799220,21cyclictest29256-21apt-get07:51:203
961799220,21cyclictest18226-21apt-get09:41:223
961799220,21cyclictest12377-21perf10:35:003
961799220,21cyclictest0-21swapper/309:35:003
9613992220,1cyclictest15267-21munin-run11:45:002
961399220,21cyclictest8609-21apt-get09:21:222
961399220,21cyclictest5247-21munin-run11:25:002
961399220,21cyclictest30346-21apt-get11:11:232
961399220,21cyclictest22821-21apt-get12:01:212
961399220,21cyclictest1641-21cron12:15:002
961399220,21cyclictest15767-21perf08:30:012
961399220,0cyclictest0-21swapper/210:56:202
961399220,0cyclictest0-21swapper/207:31:202
961099220,21cyclictest8526-21cron08:15:011
961099220,21cyclictest2864-21apt-get12:26:221
961099220,21cyclictest2855-21awk11:19:591
961099220,21cyclictest2354-21perf10:15:011
961099220,21cyclictest2349-21apt-get10:15:031
961099220,21cyclictest0-21swapper/111:56:261
961099220,21cyclictest0-21swapper/109:05:021
961099220,0cyclictest0-21swapper/107:46:201
9605992220,1cyclictest10429-21cron11:35:000
9605992215,1cyclictest0-21swapper/010:24:590
960599220,21cyclictest24872-21munin-run12:05:000
960599220,21cyclictest15154-21apt-get10:41:210
960599220,21cyclictest0-21swapper/008:11:240
960599220,1cyclictest15458-21munin-run09:34:590
9617992120,1cyclictest976-21apt-get09:06:183
9617992119,1cyclictest15611-21apt-get11:46:223
9617992117,0cyclictest0-21swapper/312:26:233
961799210,20cyclictest5234-21apt-get11:25:023
961799210,20cyclictest0-21swapper/310:36:223
961799210,20cyclictest0-21swapper/310:15:013
961799210,0cyclictest20619-21apt-get09:46:213
9613992121,0cyclictest0-21swapper/208:01:172
9613992119,1cyclictest20592-21sendmail08:40:002
961399210,21cyclictest0-21swapper/207:16:202
961399210,20cyclictest0-21swapper/209:06:182
961399210,20cyclictest0-21swapper/209:05:002
961399210,0cyclictest0-21swapper/210:41:212
961399210,0cyclictest0-21swapper/210:15:032
9610992121,0cyclictest0-21swapper/108:26:201
9610992121,0cyclictest0-21swapper/107:51:191
9610992119,1cyclictest27478-21sendmail-msp10:00:001
961099210,21cyclictest0-21swapper/109:15:021
961099210,20cyclictest0-21swapper/112:01:221
961099210,20cyclictest0-21swapper/108:56:221
961099210,20cyclictest0-21swapper/108:16:211
961099210,1cyclictest9741-21munin-plugin-st12:40:001
961099210,15cyclictest29646-21awk12:15:001
961099210,0cyclictest0-21swapper/110:26:231
961099210,0cyclictest0-21swapper/109:41:211
961099210,0cyclictest0-21swapper/107:11:201
9605992121,0cyclictest0-21swapper/008:26:190
9605992120,1cyclictest13212-21apt-get11:41:220
9605992120,1cyclictest0-21swapper/009:06:190
9605992120,1cyclictest0-21swapper/008:51:210
9605992119,1cyclictest754-21apt-get11:16:210
9605992118,2cyclictest32659-21munin-plugin-st09:05:010
960599210,21cyclictest0-21swapper/011:36:220
960599210,21cyclictest0-21swapper/011:25:010
960599210,20cyclictest0-21swapper/011:21:210
960599210,20cyclictest0-21swapper/007:36:200
960599210,1cyclictest9938-21apt-get07:11:210
960599210,1cyclictest8911-21apt-get08:16:210
960599210,1cyclictest15809-21apt-get09:36:210
960599210,0cyclictest0-21swapper/008:01:180
960599210,0cyclictest0-21swapper/007:51:200
9617992020,0cyclictest0-21swapper/312:01:213
9617992020,0cyclictest0-21swapper/311:41:223
9617992020,0cyclictest0-21swapper/308:51:213
9617992020,0cyclictest0-21swapper/308:21:213
9617992020,0cyclictest0-21swapper/308:16:203
9617992018,1cyclictest32638-21perf09:05:003
9617992018,1cyclictest25414-21apt-get11:00:023
9617992018,1cyclictest12830-21tr11:40:013
961799200,20cyclictest0-21swapper/311:21:223
961799200,0cyclictest22051-21apt-get07:36:213
961799200,0cyclictest0-21swapper/310:51:233
961799200,0cyclictest0-21swapper/310:41:223
9613992020,0cyclictest25741-21apt-get08:51:202
9613992020,0cyclictest25414-21apt-get09:56:212
9613992020,0cyclictest0-21swapper/212:26:232
9613992020,0cyclictest0-21swapper/211:26:232
9613992020,0cyclictest0-21swapper/210:01:292
9613992020,0cyclictest0-21swapper/209:11:212
9613992018,1cyclictest2492-21awk12:25:012
9613992018,1cyclictest22429-21cron11:59:592
9613992018,1cyclictest0-21swapper/212:20:012
9613992018,1cyclictest0-21swapper/210:35:002
9613992018,1cyclictest0-21swapper/210:25:002
961399200,20cyclictest0-21swapper/210:46:212
961399200,0cyclictest26862-21apt-get07:46:212
961399200,0cyclictest0-21swapper/211:31:212
961399200,0cyclictest0-21swapper/209:36:222
961399200,0cyclictest0-21swapper/207:11:212
9610992020,0cyclictest0-21swapper/112:31:231
9610992020,0cyclictest0-21swapper/111:46:221
9610992020,0cyclictest0-21swapper/110:31:261
9610992018,1cyclictest0-21swapper/107:26:221
961099200,20cyclictest0-21swapper/110:36:211
961099200,20cyclictest0-21swapper/109:01:211
961099200,19cyclictest14389-21tr07:20:001
961099200,19cyclictest0-21swapper/107:36:201
961099200,0cyclictest0-21swapper/108:01:171
9605992020,0cyclictest0-21swapper/012:16:200
9605992020,0cyclictest0-21swapper/008:21:210
9605992018,1cyclictest27830-21apt-get10:01:220
9605992018,1cyclictest24781-21apt-get11:00:030
9605992017,3cyclictest0-21swapper/012:26:220
9605992016,3cyclictest27245-21sh12:09:590
9605992014,1cyclictest593-21dbus-daemon12:25:000
960599200,20cyclictest0-21swapper/010:15:020
960599200,0cyclictest0-21swapper/010:36:210
960599200,0cyclictest0-21swapper/010:34:590
960599200,0cyclictest0-21swapper/007:26:210
9617991919,0cyclictest0-21swapper/311:16:223
9617991919,0cyclictest0-21swapper/307:26:213
9617991918,1cyclictest0-21swapper/310:56:203
9617991918,0cyclictest25214-21apt-get12:06:203
9617991917,1cyclictest593-21dbus-daemon07:45:013
9617991916,2cyclictest596-21in:imuxsock10:45:033
9617991916,2cyclictest14754-21apt-get07:21:203
9613991919,0cyclictest0-21swapper/211:36:212
9613991919,0cyclictest0-21swapper/211:00:022
9613991919,0cyclictest0-21swapper/210:11:222
9613991919,0cyclictest0-21swapper/209:41:222
9613991919,0cyclictest0-21swapper/209:26:222
9613991919,0cyclictest0-21swapper/207:56:222
9613991917,1cyclictest28-21ksoftirqd/210:55:002
9613991917,1cyclictest19170-21sh07:30:012
9610991919,0cyclictest0-21swapper/111:31:211
9610991919,0cyclictest0-21swapper/111:06:231
9610991919,0cyclictest0-21swapper/107:56:211
9610991918,1cyclictest22738-21apt-get10:56:211
9610991917,1cyclictest0-21swapper/111:54:591
9605991918,0cyclictest3394-21apt-get09:11:220
9605991918,0cyclictest0-21swapper/009:56:220
9605991918,0cyclictest0-21swapper/007:16:220
9605991917,1cyclictest0-21swapper/007:21:210
960599190,15cyclictest29995-21munin-run11:09:590
9617991818,0cyclictest12333-21apt-get07:16:213
9617991818,0cyclictest0-21swapper/310:01:223
9617991818,0cyclictest0-21swapper/309:56:223
9617991818,0cyclictest0-21swapper/309:21:203
9617991818,0cyclictest0-21swapper/309:11:213
9613991818,0cyclictest0-21swapper/208:46:202
9613991818,0cyclictest0-21swapper/207:41:242
9613991818,0cyclictest0-21swapper/207:21:212
9613991817,1cyclictest27263-21perf12:10:002
961399180,0cyclictest12736-21apt-get10:36:222
9610991818,0cyclictest23345-21apt-get08:46:201
9610991818,0cyclictest18530-21apt-get08:36:211
9610991818,0cyclictest0-21swapper/111:21:211
9610991818,0cyclictest0-21swapper/110:06:241
9610991818,0cyclictest0-21swapper/107:31:191
9610991816,1cyclictest5442-21munin-run09:15:011
961099180,17cyclictest0-21swapper/111:36:211
9605991818,0cyclictest0-21swapper/011:46:220
9605991818,0cyclictest0-21swapper/010:56:210
9605991818,0cyclictest0-21swapper/010:51:230
9605991818,0cyclictest0-21swapper/009:21:200
9605991816,1cyclictest5233-21/usr/sbin/munin12:31:290
961799172,4cyclictest593-21dbus-daemon08:50:003
9617991717,0cyclictest0-21swapper/309:16:213
9617991715,1cyclictest11013-21apt-get09:26:223
961799170,16cyclictest612-21gdbus11:14:593
961799170,16cyclictest19546-21apt-get07:31:203
9613991717,0cyclictest0-21swapper/212:31:242
9613991717,0cyclictest0-21swapper/211:51:212
9613991717,0cyclictest0-21swapper/210:26:232
9613991717,0cyclictest0-21swapper/209:31:222
9613991717,0cyclictest0-21swapper/208:16:202
9613991715,1cyclictest0-21swapper/207:54:592
9610991717,0cyclictest0-21swapper/109:46:201
9610991715,1cyclictest17848-21latency_hist09:39:591
961099170,16cyclictest29896-21idleruntime-cro10:05:001
9605991717,0cyclictest0-21swapper/010:46:230
9605991717,0cyclictest0-21swapper/009:16:220
9605991717,0cyclictest0-21swapper/007:45:000
9605991716,0cyclictest27610-21apt-get12:11:210
9617991616,0cyclictest31710-21apt-get07:56:233
9617991616,0cyclictest0-21swapper/312:11:213
9617991616,0cyclictest0-21swapper/311:06:223
9617991616,0cyclictest0-21swapper/309:36:223
9617991616,0cyclictest0-21swapper/307:11:203
9617991615,1cyclictest20409-21apt-get11:56:263
9617991614,1cyclictest0-21swapper/311:54:593
9613991613,2cyclictest16556-21df08:31:232
961399160,12cyclictest781-21lldpd09:54:012
9610991616,0cyclictest0-21swapper/111:41:221
9610991616,0cyclictest0-21swapper/108:51:201
9610991615,1cyclictest0-21swapper/110:46:211
961099160,7cyclictest0-21swapper/111:11:231
9605991616,0cyclictest0-21swapper/007:56:230
9605991614,1cyclictest8350-21df_inode10:26:230
9605991614,1cyclictest20027-21ls11:55:000
9605991614,1cyclictest0-21swapper/008:10:010
9617991515,0cyclictest20950-21apt-get08:41:223
9617991514,1cyclictest2506-21latency_hist12:25:013
9613991515,0cyclictest0-21swapper/208:41:212
961399150,14cyclictest29669-21users11:06:332
9610991515,0cyclictest0-21swapper/109:26:221
9610991515,0cyclictest0-21swapper/108:31:201
9610991514,1cyclictest5989-21apt-get11:26:221
9610991514,1cyclictest1638-21pmu-power12:21:281
9610991513,1cyclictest0-21swapper/108:06:251
961099150,14cyclictest0-21swapper/109:21:221
9605991515,0cyclictest30346-21apt-get11:11:220
9605991515,0cyclictest0-21swapper/011:56:270
9605991515,0cyclictest0-21swapper/009:41:210
9605991515,0cyclictest0-21swapper/009:26:220
9605991515,0cyclictest0-21swapper/008:41:210
9605991514,0cyclictest32725-21apt-get10:11:230
9617991414,0cyclictest0-21swapper/308:36:013
961799140,14cyclictest0-21swapper/312:34:293
961799140,14cyclictest0-21swapper/310:05:183
961799140,14cyclictest0-21swapper/308:05:113
961799140,0cyclictest0-21swapper/310:25:213
961799140,0cyclictest0-21swapper/310:12:133
9613991414,0cyclictest0-21swapper/212:36:232
9613991414,0cyclictest0-21swapper/211:45:002
9613991414,0cyclictest0-21swapper/208:55:382
961399140,14cyclictest0-21swapper/210:05:062
961399140,14cyclictest0-21swapper/209:45:262
961399140,14cyclictest0-21swapper/209:17:342
961399140,14cyclictest0-21swapper/208:10:102
961399140,14cyclictest0-21swapper/207:38:272
9610991414,0cyclictest0-21swapper/109:51:211
9610991414,0cyclictest0-21swapper/108:41:211
961099140,14cyclictest0-21swapper/112:05:431
961099140,14cyclictest0-21swapper/110:20:211
961099140,14cyclictest0-21swapper/109:34:581
9605991414,0cyclictest0-21swapper/009:45:090
9605991414,0cyclictest0-21swapper/008:36:220
960599140,14cyclictest0-21swapper/010:05:190
960599140,14cyclictest0-21swapper/007:30:010
960599140,0cyclictest0-21swapper/008:49:120
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional