You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-15 - 01:33

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack2slot7s.osadl.org (updated Fri Nov 14, 2025 12:39:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
4142805996937,10cyclictest421-21avahi-daemon10:30:010
4142805995741,5cyclictest44853-21cron10:40:010
4142805995739,14cyclictest151ktimers/011:14:150
4142806995543,8cyclictest955-21mta-sts-daemon11:24:091
4142805995343,5cyclictest0-21swapper/011:19:070
4142805995343,5cyclictest0-21swapper/011:19:070
4142805995130,14cyclictest12450irq/32-eth009:33:020
4142806994939,7cyclictest422-21dbus-daemon10:15:011
4142807994836,8cyclictest57364-21/usr/sbin/munin10:59:252
4142806994837,4cyclictest0-21swapper/109:13:021
4142805994733,11cyclictest4153318-21systemd-run07:45:010
414280599471,39cyclictest120443-21ssh12:34:090
4142806994636,7cyclictest30715-21grep10:19:141
4142805994632,11cyclictest100838-21systemd-run12:05:010
4142805994629,14cyclictest107638-21cut12:15:010
4142807994535,6cyclictest31512-21awk10:20:012
4142805994526,16cyclictest12450irq/32-eth010:09:070
4142807994336,4cyclictest42298-21kworker/u8:7-xprtiod10:40:012
4142807994335,5cyclictest18120-21latency_hist10:00:022
4142807994333,7cyclictest421-21avahi-daemon09:11:232
4142806994334,6cyclictest0-21swapper/108:04:161
4142806994333,6cyclictest14809-21latency_hist09:55:031
414280699421,35cyclictest4193040-21ssh09:33:021
414280799413,29cyclictest3313763-21cups-browsed09:09:062
4142806994133,5cyclictest4177288-21munin-plugin-st09:05:011
4142806994132,6cyclictest3313763-21cups-browsed12:15:361
4142806994132,4cyclictest100051-21ps12:04:171
4142806994129,9cyclictest76530-21ntp_kernel_pll_11:29:181
4142806994129,9cyclictest76530-21ntp_kernel_pll_11:29:181
4142805994131,7cyclictest4182664-21systemd-run09:15:020
4142805994131,5cyclictest151ktimers/008:39:100
4142805994125,12cyclictest12450irq/32-eth010:44:140
4142805994125,12cyclictest12450irq/32-eth010:44:140
4142805994124,13cyclictest60706-21unixbench_singl11:04:250
4142805994124,13cyclictest60706-21unixbench_singl11:04:250
414280799403,27cyclictest23958-21ssh10:09:062
4142807994031,6cyclictest4379-21sendmail09:40:012
4142806994033,3cyclictest0-21swapper/110:44:101
4142806994033,3cyclictest0-21swapper/110:44:101
4142806994031,5cyclictest57638-21munin-run11:00:011
4142806994029,8cyclictest698-21cron11:05:001
4142806994029,8cyclictest698-21cron11:05:001
4142806994029,8cyclictest106378-21sh12:14:011
4142805994033,5cyclictest266-21systemd-journal12:00:010
414280799394,27cyclictest60163-21ssh11:04:162
414280799394,27cyclictest60163-21ssh11:04:162
4142807993931,5cyclictest100851-21latency_hist12:05:022
4142806993933,3cyclictest0-21swapper/112:40:011
4142806993932,4cyclictest0-21swapper/112:00:011
4142806993931,4cyclictest0-21swapper/108:20:011
4142805993933,3cyclictest0-21swapper/012:30:020
414280799383,29cyclictest82793-21ssh11:39:072
4142807993831,3cyclictest0-21swapper/212:35:012
4142807993831,3cyclictest0-21swapper/212:24:132
4142807993817,18cyclictest422-21dbus-daemon11:09:182
414280699383,31cyclictest79585-21ssh11:34:141
414280699383,31cyclictest79585-21ssh11:34:141
4142806993831,4cyclictest0-21swapper/108:40:011
4142806993829,6cyclictest4174299-21systemd-run08:55:011
4142806993829,5cyclictest4175235-21/usr/sbin/munin08:59:101
4142806993825,9cyclictest4178790-21diskmemload10:49:011
4142805993830,4cyclictest151ktimers/009:44:130
4142805993827,8cyclictest0-21swapper/010:54:220
4142805993824,11cyclictest0-21swapper/011:54:180
414280599382,31cyclictest110201-21ssh12:19:060
414280799379,10cyclictest4191073-21ntpq09:29:132
4142807993729,5cyclictest36905-21fschecks_time10:29:032
4142806993730,4cyclictest89128-21aten_r2power_fr11:48:591
4142806993729,5cyclictest71100-21latency_hist11:20:021
4142806993729,5cyclictest71100-21latency_hist11:20:021
4142806993729,5cyclictest4191756-21munin-plugin-st09:30:011
4142806993729,3cyclictest0-21swapper/110:39:131
4142806993728,6cyclictest23775-21wc10:09:021
4142805993729,6cyclictest85105-21sh11:42:470
4142805993725,9cyclictest421-21avahi-daemon09:54:150
414280799368,10cyclictest5561-21rm09:42:202
4142807993631,3cyclictest4171411-21kworker/u8:7-xprtiod09:20:012
4142807993610,18cyclictest106815-21ssh12:14:092
4142806993628,5cyclictest104337-21latency_hist12:10:021
4142806993627,6cyclictest453-21systemd-logind09:50:011
4142806993625,7cyclictest3313763-21cups-browsed10:33:351
414280699362,29cyclictest20581-21ssh10:04:171
4142806993615,15cyclictest422-21dbus-daemon07:24:081
4142805993629,3cyclictest0-21swapper/011:24:150
4142805993626,7cyclictest113476-21latency12:24:050
414280799353,25cyclictest86242-21ssh11:44:122
4142807993529,3cyclictest0-21swapper/212:19:082
4142807993527,5cyclictest90693-21systemd-run11:50:012
4142807993527,5cyclictest11316-21idleruntime-cro09:50:012
4142807993526,6cyclictest70581-21unixbench_singl11:19:202
4142807993526,6cyclictest70581-21unixbench_singl11:19:202
4142807993526,6cyclictest422-21dbus-daemon09:31:142
4142807993525,6cyclictest4169779-21munin-plugin-st08:40:012
4142807993523,5cyclictest103397-21ntpq12:09:112
4142806993527,5cyclictest54374-21latency_hist10:55:021
4142806993526,6cyclictest4144309-21systemd-run07:15:021
4142806993526,6cyclictest266-21systemd-journal11:55:011
4142806993525,5cyclictest0-21swapper/109:24:121
4142806993523,8cyclictest37700-21timerandwakeup10:29:221
4142805993529,3cyclictest4175787-21plymouth09:00:010
4142805993528,4cyclictest10801-21timerandwakeup09:49:200
4142805993527,6cyclictest75847-21munin-node11:28:580
4142805993527,6cyclictest75847-21munin-node11:28:580
4142805993527,5cyclictest18008-21cron10:00:010
414280599352,27cyclictest4187729-21ssh09:24:090
4142805993522,10cyclictest0-21swapper/010:25:010
4142807993427,4cyclictest4138817-21kworker/u8:1+nfsiod08:00:012
4142807993425,6cyclictest125020-21cron12:40:012
4142807993425,4cyclictest0-21swapper/207:24:082
4142807993424,6cyclictest75951-21aten_r2power_fr11:29:072
4142807993424,6cyclictest75951-21aten_r2power_fr11:29:072
4142807993423,6cyclictest21361-21idleruntime-cro10:05:012
4142807993422,9cyclictest4187248-21cpu09:24:002
4142807993422,9cyclictest4155540-21missed_timers07:54:042
414280699345,22cyclictest63449-21ssh11:09:161
414280699344,24cyclictest959-21mta-sts-daemon07:19:151
414280699344,20cyclictest4178790-21diskmemload12:24:081
4142806993426,5cyclictest4142827-21latency_hist07:10:011
4142806993426,4cyclictest0-21swapper/110:24:061
414280699342,27cyclictest120381-21ssh12:34:061
4142805993417,15cyclictest4563-21latency_hist09:40:030
414280799338,7cyclictest91979-21sh11:52:572
414280799337,8cyclictest96587-21ntpq11:59:092
4142807993326,3cyclictest0-21swapper/210:44:102
4142807993326,3cyclictest0-21swapper/210:44:102
414280699335,22cyclictest7052-21ssh09:44:121
414280599339,18cyclictest56730-21ssh10:59:140
4142805993325,5cyclictest3313763-21cups-browsed11:38:540
4142805993325,5cyclictest20209-21latency10:04:060
4142805993323,8cyclictest4156279-21sh07:55:010
4142805993313,12cyclictest151ktimers/008:04:140
414280799328,8cyclictest50578-21ssh10:49:152
414280799324,21cyclictest73359-21ssh11:24:122
4142807993225,4cyclictest4144186-21unixbench_multi07:14:182
4142807993224,5cyclictest4166811-21systemd-run08:30:022
4142807993220,9cyclictest33144-21kworker/u8:1+nfsiod10:24:112
4142807993219,8cyclictest4154276-21ntpq07:49:112
4142807993210,7cyclictest64730-21sh11:10:352
414280699326,18cyclictest117228-21ssh12:29:161
4142806993227,2cyclictest0-21swapper/109:20:011
4142806993225,5cyclictest4178910-21latency_hist09:10:031
4142806993225,4cyclictest955-21mta-sts-daemon09:55:381
4142806993223,6cyclictest474-21in:imuxsock08:15:011
4142806993223,6cyclictest421-21avahi-daemon07:45:021
4142806993223,5cyclictest83708-21cron11:40:001
414280699322,25cyclictest86318-21ssh11:44:141
4142805993224,5cyclictest7859-21kworker/u8:23+xprtiod11:09:240
4142805993221,3cyclictest4177384-21kworker/u8:12-xprtiod09:10:010
4142805993211,15cyclictest12450irq/32-eth009:29:070
414280799317,7cyclictest4170275-21cpuspeed08:44:002
414280799314,18cyclictest0-21swapper/210:14:122
4142807993125,3cyclictest0-21swapper/208:55:012
4142807993119,8cyclictest453-21systemd-logind08:35:012
4142807993119,8cyclictest4176731-21ntpq09:04:092
4142807993118,8cyclictest4158734-21ntpq08:04:172
4142807993118,8cyclictest4149653-21ntpq07:34:062
4142807993118,8cyclictest4148191-21ntpq07:29:092
4142806993123,5cyclictest4150282-21idleruntime-cro07:35:011
414280699312,23cyclictest66744-21ssh11:14:081
4142805993126,2cyclictest0-21swapper/011:49:090
4142805993123,5cyclictest102649-21aten_r2power_en12:08:570
414280799308,4cyclictest36-21ksoftirqd/209:50:242
4142807993022,5cyclictest40049-21cpu10:34:032
4142807993018,8cyclictest53449-21ntpq10:54:112
4142807993018,8cyclictest4161646-21ntpq08:14:062
4142807993017,8cyclictest4164763-21ntpq08:24:112
4142807993015,11cyclictest78005-21ssh11:31:262
4142807993015,11cyclictest78005-21ssh11:31:262
414280599302,23cyclictest12450irq/32-eth008:14:090
4142805993018,9cyclictest80465-21ssh11:35:010
4142805993018,9cyclictest80465-21ssh11:35:010
4142805993018,9cyclictest0-21swapper/007:35:010
4142805993017,9cyclictest12450irq/32-eth008:19:130
4142807992917,7cyclictest4172226-21ntpq08:49:132
4142807992917,7cyclictest4172226-21ntpq08:49:132
4142807992916,9cyclictest117109-21ntpq12:29:102
4142807992916,8cyclictest4163270-21ntpq08:19:122
4142807992915,8cyclictest4145239-21ntpq07:19:122
4142805992925,2cyclictest0-21swapper/007:40:010
4142805992923,4cyclictest30129-21uniq10:19:010
4142805992921,6cyclictest4166125-21needreboot08:29:090
4142805992917,4cyclictest4168270-21cron08:35:010
4142806992822,3cyclictest0-21swapper/107:40:011
4142806992822,3cyclictest0-21swapper/107:29:101
4142806992821,4cyclictest957-21mta-sts-daemon07:49:121
4142806992820,5cyclictest422-21dbus-daemon08:24:201
4142806992820,5cyclictest421-21avahi-daemon07:54:151
4142806992820,5cyclictest4159844-21fschecks_time08:09:011
4142806992820,5cyclictest4159844-21fschecks_time08:09:011
4142806992818,7cyclictest959-21mta-sts-daemon08:34:061
4142805992822,4cyclictest4176347-21fschecks_time09:04:010
4142805992821,4cyclictest40210-21/usr/sbin/munin10:34:060
4142805992821,3cyclictest0-21swapper/009:19:340
4142805992818,3cyclictest0-21swapper/010:49:060
4142807992716,7cyclictest4151195-21ntpq07:39:112
4142807992715,8cyclictest4175210-21ntpq08:59:102
4142806992718,7cyclictest4171239-21unixbench_singl08:44:201
4142806992716,8cyclictest1-21systemd08:30:021
414280699271,20cyclictest3496-21ssh09:39:111
4142805992720,4cyclictest1-21systemd07:50:010
4142805992717,8cyclictest121707-21sshd12:35:280
4142805992714,10cyclictest0-21swapper/010:14:100
414280599271,21cyclictest3313742-21cupsd07:59:060
4142807992616,7cyclictest4152791-21ntpq07:44:112
4142807992613,8cyclictest4160155-21ntpq08:09:052
4142807992613,8cyclictest4160155-21ntpq08:09:052
4142805992621,2cyclictest0-21swapper/008:50:010
4142805992621,2cyclictest0-21swapper/008:50:010
4142806992511,10cyclictest3313763-21cups-browsed07:59:061
4142805992520,3cyclictest0-21swapper/007:19:110
4142805992520,2cyclictest0-21swapper/007:24:190
4142805992519,3cyclictest957-21mta-sts-daemon07:14:010
4142805992516,6cyclictest4148801-21awk07:30:010
4142805992416,6cyclictest4173470-21latency08:54:120
4142807992313,7cyclictest4142817-21munin-run07:10:012
4142806992318,3cyclictest0-21swapper/108:49:111
4142806992318,3cyclictest0-21swapper/108:49:111
414280599232,15cyclictest4170656-21ntpq08:44:070
414280599231,15cyclictest151ktimers/008:24:110
414280899212,11cyclictest0-21swapper/309:34:083
4142805992115,4cyclictest959-21mta-sts-daemon08:09:050
4142805992115,4cyclictest959-21mta-sts-daemon08:09:050
414280899183,7cyclictest0-21swapper/309:54:113
414280599181,6cyclictest266-21systemd-journal07:10:010
414280899172,7cyclictest0-21swapper/312:24:083
414280899162,7cyclictest0-21swapper/310:04:123
414280899162,7cyclictest0-21swapper/307:59:063
414280899162,5cyclictest0-21swapper/312:33:563
414280899161,6cyclictest0-21swapper/308:54:203
414280899157,5cyclictest0-21swapper/311:54:113
414280899152,7cyclictest0-21swapper/311:14:153
414280899152,7cyclictest0-21swapper/309:13:043
414280899152,6cyclictest0-21swapper/310:29:173
414280899152,5cyclictest0-21swapper/307:34:103
414280899151,7cyclictest0-21swapper/311:49:133
414280899151,7cyclictest0-21swapper/310:49:063
414280899151,7cyclictest0-21swapper/310:44:083
414280899151,7cyclictest0-21swapper/310:44:083
414280899151,7cyclictest0-21swapper/308:14:073
414280899143,6cyclictest0-21swapper/310:54:113
414280899143,6cyclictest0-21swapper/309:59:103
414280899143,4cyclictest0-21swapper/307:39:093
414280899142,6cyclictest0-21swapper/309:24:143
414280899142,5cyclictest0-21swapper/308:59:103
414280899141,7cyclictest0-21swapper/310:24:113
414280899141,7cyclictest0-21swapper/309:44:103
414280899141,7cyclictest0-21swapper/309:39:083
414280899141,7cyclictest0-21swapper/309:18:283
414280899141,6cyclictest0-21swapper/310:09:003
414280899141,5cyclictest0-21swapper/309:09:053
414280899134,6cyclictest0-21swapper/312:09:093
414280899133,5cyclictest0-21swapper/311:30:013
414280899133,5cyclictest0-21swapper/311:30:013
414280899132,7cyclictest0-21swapper/310:39:093
414280899132,6cyclictest0-21swapper/311:09:143
414280899132,6cyclictest0-21swapper/310:59:103
414280899132,6cyclictest0-21swapper/310:34:083
414280899132,5cyclictest0-21swapper/312:36:283
414280899132,5cyclictest0-21swapper/312:03:583
414280899132,5cyclictest0-21swapper/311:58:593
414280899131,7cyclictest0-21swapper/312:19:063
414280899131,7cyclictest0-21swapper/311:44:163
414280899131,7cyclictest0-21swapper/310:14:113
414280899131,6cyclictest0-21swapper/311:34:103
414280899131,6cyclictest0-21swapper/311:34:103
414280899131,6cyclictest0-21swapper/311:19:073
414280899131,6cyclictest0-21swapper/311:19:073
414280899131,6cyclictest0-21swapper/308:34:123
414280899131,6cyclictest0-21swapper/308:20:283
414280899131,6cyclictest0-21swapper/307:19:113
414280899131,5cyclictest0-21swapper/308:39:083
414280899131,5cyclictest0-21swapper/308:09:073
414280899131,5cyclictest0-21swapper/308:09:073
414280899131,5cyclictest0-21swapper/307:44:113
414280899131,5cyclictest0-21swapper/307:29:143
414280899122,6cyclictest0-21swapper/309:49:013
414280899121,7cyclictest0-21swapper/309:29:073
414280899121,6cyclictest0-21swapper/312:14:013
414280899121,6cyclictest0-21swapper/311:24:043
414280899121,6cyclictest0-21swapper/308:04:153
414280899121,6cyclictest0-21swapper/307:24:063
414280899121,5cyclictest0-21swapper/308:29:123
414280899121,5cyclictest0-21swapper/307:54:083
414280899120,6cyclictest0-21swapper/310:18:583
414280899120,6cyclictest0-21swapper/307:49:093
414280899111,6cyclictest0-21swapper/312:29:013
414280899111,6cyclictest0-21swapper/311:38:573
414280899111,5cyclictest0-21swapper/308:49:103
414280899111,5cyclictest0-21swapper/308:49:103
414280899111,5cyclictest0-21swapper/308:44:103
414280899111,5cyclictest0-21swapper/308:17:283
414280899110,5cyclictest0-21swapper/307:14:053
414280899102,4cyclictest0-21swapper/311:03:513
414280899102,4cyclictest0-21swapper/311:03:513
41428089991,5cyclictest0-21swapper/309:03:563
41428089981,4cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional