You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-22 - 20:38

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Sat Nov 22, 2025 12:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
7989992423,1cyclictest12695-21apt-get09:32:132
7986992422,1cyclictest1841-21cron10:15:011
7986992422,1cyclictest12272-21basename09:30:001
7989992323,0cyclictest0-21swapper/210:47:122
7989992321,1cyclictest0-21swapper/212:35:002
798999230,22cyclictest27691-21apt12:15:002
7982992320,2cyclictest30096-21wc12:20:000
798299230,22cyclictest1155-21sendmail11:20:000
7992992222,0cyclictest3985-21apt-get11:27:123
7992992221,0cyclictest16782-21apt-get10:47:133
7992992220,1cyclictest31162-21grep11:15:003
7992992220,1cyclictest19806-21munin-run08:40:003
7992992220,1cyclictest0-21swapper/310:20:003
799299220,21cyclictest9866-21sh09:25:013
799299220,21cyclictest15088-21apt-get09:37:103
799299220,21cyclictest0-21swapper/310:52:113
799299220,21cyclictest0-21swapper/308:10:023
799299220,21cyclictest0-21swapper/307:42:123
799299220,1cyclictest0-21swapper/308:35:013
7989992220,1cyclictest7361-21cron09:20:012
7989992220,1cyclictest20300-21apt-get09:47:132
798999220,21cyclictest27818-21apt-get08:57:122
798999220,21cyclictest23920-21cron11:00:012
798999220,21cyclictest0-21swapper/211:57:112
798999220,1cyclictest25420-21apt-get08:52:122
798999220,1cyclictest24391-21apt-get11:02:132
798699220,21cyclictest2947-21apt-get08:07:111
798699220,21cyclictest29181-21apt-get11:12:121
798699220,21cyclictest21967-21apt-get10:57:101
798699220,21cyclictest17900-21apt-get09:42:101
798699220,21cyclictest10832-21sendmail-msp11:39:591
798699220,21cyclictest10625-21apt-get08:22:131
798699220,21cyclictest0-21swapper/112:32:111
798699220,21cyclictest0-21swapper/107:12:111
7982992220,1cyclictest0-21swapper/012:40:010
7982992220,1cyclictest0-21swapper/010:50:000
798299220,21cyclictest6382-21apt-get11:32:100
798299220,21cyclictest31167-21munin-run11:15:000
798299220,21cyclictest0-21swapper/011:42:110
798299220,21cyclictest0-21swapper/007:50:010
7992992120,1cyclictest27511-21apt-get10:02:103
7992992120,0cyclictest0-21swapper/308:02:113
7992992119,1cyclictest13253-21munin-plugin-st11:45:013
799299210,20cyclictest0-21swapper/312:27:123
799299210,20cyclictest0-21swapper/308:52:113
799299210,1cyclictest4581-21sh09:15:003
799299210,0cyclictest0-21swapper/307:17:123
7989992121,0cyclictest30532-21apt-get12:22:122
7989992120,0cyclictest25094-21apt-get09:57:122
7989992119,1cyclictest7318-21munin-run12:40:012
7989992119,1cyclictest17063-21apt-get09:40:012
798999210,2cyclictest10814-21sh11:40:002
798999210,21cyclictest0-21swapper/210:22:132
798999210,20cyclictest29911-21apt-get10:07:142
798999210,20cyclictest0-21swapper/211:32:112
798999210,20cyclictest0-21swapper/208:37:112
798999210,20cyclictest0-21swapper/207:57:112
798999210,20cyclictest0-21swapper/207:37:122
798999210,1cyclictest0-21swapper/208:42:112
798999210,0cyclictest15640-21apt-get07:27:122
798999210,0cyclictest0-21swapper/210:52:112
7986992121,0cyclictest0-21swapper/110:17:151
7986992120,1cyclictest0-21swapper/109:52:111
7986992119,1cyclictest7110-21apt-get10:27:131
7986992119,1cyclictest5211-21apt-get09:15:021
798699210,21cyclictest0-21swapper/111:02:131
798699210,21cyclictest0-21swapper/110:32:111
798699210,20cyclictest0-21swapper/108:47:111
798699210,0cyclictest30234-21apt-get09:02:121
798699210,0cyclictest0-21swapper/111:27:121
798699210,0cyclictest0-21swapper/107:27:121
7982992121,0cyclictest0-21swapper/012:22:130
7982992121,0cyclictest0-21swapper/009:27:100
7982992121,0cyclictest0-21swapper/007:27:110
7982992121,0cyclictest0-21swapper/007:17:120
7982992120,1cyclictest13018-21apt-get08:27:110
7982992120,1cyclictest0-21swapper/008:09:590
7982992119,1cyclictest4255-21grep10:19:590
7982992119,1cyclictest0-21swapper/008:50:000
7982992119,1cyclictest0-21swapper/008:00:000
798299210,21cyclictest4909-21perf12:34:590
798299210,21cyclictest28145-21apt-get07:52:140
798299210,21cyclictest0-21swapper/010:37:130
798299210,20cyclictest17815-21apt-get08:37:120
798299210,20cyclictest0-21swapper/010:52:110
798299210,20cyclictest0-21swapper/010:42:120
798299210,20cyclictest0-21swapper/009:42:100
798299210,20cyclictest0-21swapper/008:42:110
798299210,20cyclictest0-21swapper/007:22:090
798299210,0cyclictest0-21swapper/011:27:100
7992992020,0cyclictest32304-21apt-get10:12:123
7992992020,0cyclictest0-21swapper/311:32:113
7992992020,0cyclictest0-21swapper/310:57:093
7992992020,0cyclictest0-21swapper/309:27:103
7992992020,0cyclictest0-21swapper/307:12:103
7992992018,1cyclictest32509-21ls12:25:003
7992992018,1cyclictest30124-21munin-plugin-st07:54:593
799299200,0cyclictest11971-21apt-get10:37:123
799299200,0cyclictest0-21swapper/311:50:013
799299200,0cyclictest0-21swapper/307:47:113
7989992020,0cyclictest0-21swapper/211:22:122
7989992020,0cyclictest0-21swapper/210:27:132
7989992019,1cyclictest8424-21apt-get07:12:102
7989992019,1cyclictest7834-21apt-get09:22:112
7989992019,0cyclictest4927-21perf08:10:002
7989992019,0cyclictest0-21swapper/208:22:152
7989992018,1cyclictest15217-21wc07:25:002
7989992017,2cyclictest1155-21sendmail-msp11:20:002
798999200,20cyclictest0-21swapper/211:27:122
798999200,0cyclictest0-21swapper/211:47:142
798999200,0cyclictest0-21swapper/211:12:122
7986992020,0cyclictest0-21swapper/111:32:111
7986992020,0cyclictest0-21swapper/110:02:111
7986992020,0cyclictest0-21swapper/109:47:141
7986992020,0cyclictest0-21swapper/109:12:121
7986992020,0cyclictest0-21swapper/108:57:121
7986992020,0cyclictest0-21swapper/107:37:121
7986992018,1cyclictest0-21swapper/110:10:011
7982992020,0cyclictest497-21apt-get08:02:110
7982992020,0cyclictest1579-21apt-get11:22:110
7982992020,0cyclictest0-21swapper/012:12:120
7982992020,0cyclictest0-21swapper/012:07:110
7982992020,0cyclictest0-21swapper/011:07:130
7982992020,0cyclictest0-21swapper/010:27:130
7982992020,0cyclictest0-21swapper/009:15:010
7982992020,0cyclictest0-21swapper/007:30:020
7982992018,1cyclictest141rcuc/012:00:000
7982992017,2cyclictest29817-21munin-plugin-st09:00:000
798299200,0cyclictest2622-21apt-get09:12:120
798299200,0cyclictest0-21swapper/008:52:120
7992991919,0cyclictest30234-21apt-get09:02:103
7992991919,0cyclictest0-21swapper/311:17:143
7992991919,0cyclictest0-21swapper/310:07:133
7992991919,0cyclictest0-21swapper/308:22:153
7992991918,0cyclictest0-21swapper/311:02:123
7989991919,0cyclictest23024-21apt-get08:47:112
7989991919,0cyclictest0-21swapper/210:17:152
7989991917,1cyclictest9574-21apt-get10:32:122
7989991917,1cyclictest29497-21munin-run10:05:002
7986991919,0cyclictest20626-21apt-get08:42:121
7986991919,0cyclictest15415-21apt-get08:32:111
7986991919,0cyclictest0-21swapper/112:27:121
7986991919,0cyclictest0-21swapper/110:42:121
7986991919,0cyclictest0-21swapper/109:57:121
7986991919,0cyclictest0-21swapper/109:37:091
7986991919,0cyclictest0-21swapper/109:32:121
7986991919,0cyclictest0-21swapper/107:30:011
7986991918,1cyclictest12806-21awk07:20:001
7986991918,0cyclictest0-21swapper/108:27:111
7986991917,1cyclictest7718-21turbostat08:15:001
798699190,19cyclictest0-21swapper/109:07:131
7982991919,0cyclictest20300-21apt-get09:47:130
7982991919,0cyclictest0-21swapper/011:02:130
7982991919,0cyclictest0-21swapper/010:22:120
7982991919,0cyclictest0-21swapper/008:17:140
7982991918,0cyclictest0-21swapper/009:52:110
798299190,18cyclictest219712sleep010:57:090
7992991818,0cyclictest8205-21apt-get08:17:153
7992991818,0cyclictest0-21swapper/310:27:133
7992991818,0cyclictest0-21swapper/309:57:113
7992991818,0cyclictest0-21swapper/307:22:103
7992991817,0cyclictest2947-21apt-get08:07:113
7992991816,1cyclictest18066-21idleruntime-cro11:55:003
7992991816,1cyclictest14386-21apt-get10:42:123
7989991818,0cyclictest0-21swapper/209:02:112
7989991816,1cyclictest0-21swapper/212:02:102
7989991816,1cyclictest0-21swapper/207:32:102
7986991818,0cyclictest0-21swapper/112:12:121
7986991818,0cyclictest0-21swapper/111:42:101
7986991818,0cyclictest0-21swapper/110:47:131
7986991818,0cyclictest0-21swapper/109:22:111
7986991818,0cyclictest0-21swapper/108:52:121
7986991818,0cyclictest0-21swapper/108:37:121
7986991817,0cyclictest0-21swapper/108:17:151
7986991816,1cyclictest593-21dbus-daemon07:25:001
7986991816,1cyclictest30110-21perf07:55:001
7982991818,0cyclictest8847-21apt-get11:37:120
7982991818,0cyclictest0-21swapper/009:57:120
7982991816,1cyclictest8424-21apt-get07:12:110
7982991816,1cyclictest32205-21apt09:05:000
798299180,18cyclictest0-21swapper/008:22:150
7992991717,0cyclictest0-21swapper/312:12:123
7992991717,0cyclictest0-21swapper/311:37:123
7992991717,0cyclictest0-21swapper/310:32:123
7992991717,0cyclictest0-21swapper/307:57:113
7989991717,0cyclictest26787-21apt-get11:07:132
7989991717,0cyclictest16107-21apt-get11:52:102
7989991717,0cyclictest0-21swapper/211:45:012
7989991717,0cyclictest0-21swapper/209:37:102
7989991717,0cyclictest0-21swapper/209:07:132
7989991717,0cyclictest0-21swapper/208:12:122
7989991717,0cyclictest0-21swapper/207:52:142
7989991717,0cyclictest0-21swapper/207:47:102
7989991715,1cyclictest593-21dbus-daemon12:20:002
7986991717,0cyclictest28120-21apt-get12:17:161
7986991717,0cyclictest0-21swapper/112:22:131
7986991717,0cyclictest0-21swapper/111:22:121
7986991717,0cyclictest0-21swapper/110:22:131
7986991717,0cyclictest0-21swapper/107:47:111
7986991717,0cyclictest0-21swapper/107:42:121
798699170,2cyclictest13025-21/usr/sbin/munin10:37:171
7982991717,0cyclictest0-21swapper/011:52:120
7982991717,0cyclictest0-21swapper/007:42:110
7992991616,0cyclictest0-21swapper/312:17:173
7992991616,0cyclictest0-21swapper/309:52:103
7992991616,0cyclictest0-21swapper/309:47:133
7992991616,0cyclictest0-21swapper/308:47:103
7992991616,0cyclictest0-21swapper/308:42:113
7992991616,0cyclictest0-21swapper/307:32:103
7992991615,1cyclictest2917-21apt-get12:32:113
7992991615,1cyclictest0-21swapper/311:57:123
7992991614,1cyclictest7314-21perf12:40:013
7992991614,1cyclictest29796-21awk08:59:593
7992991614,1cyclictest14674-21awk09:35:003
7989991616,0cyclictest0-21swapper/209:12:122
7989991616,0cyclictest0-21swapper/208:02:112
7989991614,1cyclictest23351-21apt-get07:42:112
7986991616,0cyclictest18506-21apt-get11:57:121
7986991616,0cyclictest0-21swapper/112:07:111
7986991613,2cyclictest1753-21ps08:02:171
7982991616,0cyclictest0-21swapper/012:27:120
7982991615,1cyclictest20935-21apt-get07:37:120
7982991615,1cyclictest0-21swapper/009:22:100
7982991614,1cyclictest0-21swapper/009:10:000
798299160,1cyclictest0-21swapper/011:47:220
798299160,15cyclictest0-21swapper/009:37:100
7992991515,0cyclictest0-21swapper/307:27:123
7989991515,0cyclictest0-21swapper/208:27:112
7986991515,0cyclictest0-21swapper/111:17:141
7982991515,0cyclictest4908-21apt-get08:10:010
7982991515,0cyclictest0-21swapper/010:12:120
7982991515,0cyclictest0-21swapper/009:32:130
7992991414,0cyclictest0-21swapper/312:00:093
7992991414,0cyclictest0-21swapper/311:20:573
7992991414,0cyclictest0-21swapper/311:05:163
7992991414,0cyclictest0-21swapper/309:40:013
7992991414,0cyclictest0-21swapper/308:26:003
7992991414,0cyclictest0-21swapper/307:37:123
7992991413,0cyclictest0-21swapper/309:15:023
799299140,0cyclictest0-21swapper/310:20:153
799299140,0cyclictest0-21swapper/309:08:143
7989991414,0cyclictest0-21swapper/212:26:042
7989991414,0cyclictest0-21swapper/207:15:262
798999140,14cyclictest0-21swapper/210:35:132
798999140,14cyclictest0-21swapper/209:51:162
798999140,14cyclictest0-21swapper/209:26:242
798999140,14cyclictest0-21swapper/208:30:482
798999140,14cyclictest0-21swapper/208:19:292
798999140,0cyclictest0-21swapper/212:06:562
7986991414,0cyclictest0-21swapper/111:50:551
7986991414,0cyclictest0-21swapper/111:45:101
798699140,14cyclictest0-21swapper/112:00:101
798699140,14cyclictest0-21swapper/107:55:361
798699140,0cyclictest0-21swapper/112:36:001
7982991414,0cyclictest0-21swapper/010:30:370
7982991414,0cyclictest0-21swapper/010:06:090
7982991414,0cyclictest0-21swapper/010:02:110
798299140,14cyclictest0-21swapper/012:02:070
798299140,14cyclictest0-21swapper/008:33:510
7986991313,0cyclictest0-21swapper/110:50:161
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional