You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-30 - 00:34

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack2slot7s.osadl.org (updated Sat Nov 29, 2025 12:39:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
579110996548,13cyclictest718649-21munin-run11:40:010
579112996046,9cyclictest3987647-21cupsd10:14:302
579110995643,7cyclictest151ktimers/012:09:200
579112995545,7cyclictest3987650-21cups-browsed07:14:202
579112995539,8cyclictest0-21swapper/210:04:182
579111994939,7cyclictest3987650-21cups-browsed12:08:371
579112994840,5cyclictest422-21dbus-daemon12:20:022
579111994838,6cyclictest266-21systemd-journal11:50:011
579110994828,15cyclictest730117-21ssh11:57:190
57911199472,37cyclictest662741-21ssh10:19:181
57911199472,37cyclictest662741-21ssh10:19:181
579111994636,7cyclictest718695-21systemd-run11:40:021
579110994637,6cyclictest607595-21systemd-run08:45:020
579110994635,5cyclictest0-21swapper/009:04:180
57911099462,36cyclictest687014-21ssh10:54:210
579112994536,5cyclictest701431-21munin-run11:15:012
579111994537,4cyclictest691235-21kworker/u8:7-rpciod11:57:201
579112994434,6cyclictest3987650-21cups-browsed09:19:552
579112994433,8cyclictest638958-21timerwakeupswit09:44:272
579111994438,3cyclictest0-21swapper/109:00:011
579110994432,8cyclictest674018-21cron10:35:000
579110994432,8cyclictest674018-21cron10:35:000
579112994335,5cyclictest632656-21systemd-run09:35:012
579112994334,6cyclictest711675-21grep11:30:012
579112994333,4cyclictest0-21swapper/212:34:162
579112994333,4cyclictest0-21swapper/212:34:162
579111994329,10cyclictest266-21systemd-journal09:58:121
579110994318,8cyclictest648663-21ntpq09:59:190
579111994233,6cyclictest653206-21systemd-run10:05:011
579111994230,7cyclictest422-21dbus-daemon10:24:251
579111994230,7cyclictest422-21dbus-daemon10:24:251
579110994233,6cyclictest0-21swapper/012:14:120
579110994232,7cyclictest619107-21systemd-run09:15:010
579110994215,8cyclictest726291-21sh11:51:420
579112994133,5cyclictest704862-21munin-run11:20:012
579112994132,6cyclictest680890-21true10:45:012
579112994130,8cyclictest718649-21munin-run11:40:012
579112994130,8cyclictest594078-21munin-run08:00:012
579111994131,6cyclictest750437-21sendmail_mailtr12:24:271
579110994130,8cyclictest12450irq/32-eth012:29:170
579110994115,18cyclictest0-21swapper/010:19:240
579110994115,18cyclictest0-21swapper/010:19:240
579110994115,15cyclictest0-21swapper/007:34:150
579112994031,6cyclictest1-21systemd08:20:012
579112994029,8cyclictest603097-21sh08:30:012
579112994029,8cyclictest603097-21sh08:30:012
579112994029,6cyclictest708271-21latency_hist11:25:022
579111994032,5cyclictest670520-21munin-plugin-st10:30:001
579111994032,5cyclictest670520-21munin-plugin-st10:30:001
579111994032,5cyclictest625855-21cron09:25:011
579111994031,6cyclictest684406-21munin-run10:50:011
579111994031,6cyclictest3987650-21cups-browsed07:19:281
579111994029,7cyclictest674106-21latency_hist10:35:021
579111994029,7cyclictest674106-21latency_hist10:35:021
57911199402,33cyclictest746496-21ssh12:19:221
579110994033,4cyclictest747399-21sendmail-msp12:20:010
579110994029,7cyclictest608563-21ntpq08:49:230
579110994027,8cyclictest598011-21ntpq08:14:210
579110994027,8cyclictest589009-21ntpq07:44:170
579110994027,8cyclictest589009-21ntpq07:44:170
579110994025,9cyclictest683671-21ntpq10:49:250
579110994024,12cyclictest652329-21ntpq10:04:190
579110994017,7cyclictest596493-21ntpq08:09:190
579112993930,6cyclictest607595-21systemd-run08:45:022
579112993919,14cyclictest351ktimers/208:49:222
579111993928,6cyclictest422-21dbus-daemon11:15:011
57911099393,28cyclictest757944-21ssh12:34:210
57911099393,28cyclictest757944-21ssh12:34:210
579110993925,9cyclictest580036-21ntpq07:14:180
579110993924,8cyclictest599558-21ntpq08:19:210
57911099392,29cyclictest703954-21ssh11:19:160
579110993915,5cyclictest0-21swapper/010:56:480
579110993913,12cyclictest748317-21ssh12:21:300
579112993829,6cyclictest636088-21latency_hist09:40:022
579112993829,6cyclictest266-21systemd-journal10:35:002
579112993829,6cyclictest266-21systemd-journal10:35:002
579112993828,7cyclictest491-21gdbus11:09:262
579112993826,8cyclictest738611-21latency12:09:142
579111993830,6cyclictest0-21swapper/111:55:011
579111993830,5cyclictest721847-21unixbench-2d11:44:311
579111993827,7cyclictest421-21avahi-daemon12:28:471
579110993831,4cyclictest694687-21systemd-run11:05:010
579110993830,4cyclictest0-21swapper/011:34:300
579110993828,7cyclictest660124-21munin-plugin-st10:15:010
579110993828,4cyclictest609989-21ntpq08:54:250
579110993827,5cyclictest666199-21ntpq10:24:160
579110993827,5cyclictest666199-21ntpq10:24:160
579110993826,8cyclictest595005-21ntpq08:04:180
579110993825,9cyclictest676544-21ntpq10:39:180
579110993825,9cyclictest676544-21ntpq10:39:180
579110993825,8cyclictest707282-21ntpq11:24:150
579110993825,8cyclictest601072-21ntpq08:24:260
579110993825,8cyclictest581520-21ntpq07:19:180
579110993814,9cyclictest644106-21sh09:53:200
579112993729,5cyclictest725507-21sh11:50:022
579112993729,4cyclictest0-21swapper/207:45:012
579112993729,4cyclictest0-21swapper/207:45:012
57911199377,22cyclictest697202-21sh11:09:181
579111993729,5cyclictest959-21mta-sts-daemon12:04:181
579110993727,4cyclictest0-21swapper/009:24:160
579110993725,8cyclictest635144-21ntpq09:39:180
579110993725,5cyclictest587460-21ntpq07:39:190
579110993724,8cyclictest614523-21ntpq09:09:200
57911099372,29cyclictest605501-21kworker/u8:508:39:190
579110993721,13cyclictest680001-21ntpq10:44:190
579110993720,14cyclictest695331-21ssh11:06:190
579112993630,3cyclictest0-21swapper/212:39:232
579112993630,3cyclictest0-21swapper/212:39:232
579112993627,3cyclictest0-21swapper/212:14:212
579111993625,8cyclictest714669-21timerandwakeup11:34:341
579110993624,7cyclictest669239-21df_inode10:29:110
579110993624,7cyclictest669239-21df_inode10:29:110
579110993623,8cyclictest710804-21ntpq11:29:200
579110993623,11cyclictest629847-21ssh09:30:450
579110993622,8cyclictest603949-21ntpq08:34:150
579110993622,11cyclictest735560-21ntpq12:04:210
579110993622,11cyclictest699340-21ssh11:13:020
579110993621,13cyclictest626169-21ssh09:25:140
579110993621,12cyclictest593525-21ntpq07:59:190
579110993620,8cyclictest620907-21ssh09:18:570
579110993616,4cyclictest719692-21sh11:41:560
579112993526,5cyclictest750395-21sendmail_mailqu12:24:252
579112993524,8cyclictest727763-21latency11:54:122
579112993521,5cyclictest957-21mta-sts-daemon10:27:212
579112993521,5cyclictest957-21mta-sts-daemon10:27:212
579111993527,5cyclictest708198-21idleruntime-cro11:25:011
579111993524,7cyclictest422-21dbus-daemon09:47:481
579110993526,4cyclictest592066-21ntpq07:54:210
579110993523,8cyclictest761978-21ntpq12:39:220
579110993523,8cyclictest761978-21ntpq12:39:220
579110993522,8cyclictest602510-21ntpq08:29:200
579110993522,8cyclictest602510-21ntpq08:29:200
579110993520,11cyclictest723494-21ssh11:48:070
579110993519,13cyclictest653699-21ssh10:05:420
579112993425,6cyclictest3987650-21cups-browsed11:34:292
579112993425,6cyclictest3987650-21cups-browsed08:39:132
579112993424,6cyclictest735301-21ssh12:04:142
579112993422,9cyclictest266-21systemd-journal09:56:122
57911299342,27cyclictest618172-21ssh09:14:162
57911199346,19cyclictest437-21mta-sts-daemon08:14:231
57911199343,25cyclictest693872-21ssh11:04:171
579111993426,5cyclictest615074-21diskmemload09:39:321
579111993425,6cyclictest594074-21ls08:00:011
579111993423,5cyclictest0-21swapper/112:33:551
579111993423,5cyclictest0-21swapper/112:33:551
579111993421,9cyclictest1-21systemd09:29:081
579110993424,6cyclictest611561-21ntpq08:59:220
579110993423,7cyclictest584102-21cpuspeed07:29:080
579110993422,8cyclictest590495-21ntpq07:49:160
579110993419,11cyclictest636845-21ssh09:41:370
579112993327,3cyclictest728778-21kworker/u8:1+xprtiod11:57:172
579112993327,3cyclictest0-21swapper/207:40:012
579112993325,5cyclictest690893-21unixbench_multi10:59:292
579112993325,5cyclictest663026-21mailstats10:19:272
579112993325,5cyclictest663026-21mailstats10:19:272
579111993325,5cyclictest421-21avahi-daemon10:39:101
579111993325,5cyclictest421-21avahi-daemon10:39:101
579111993323,5cyclictest0-21swapper/109:54:201
579111993323,3cyclictest0-21swapper/109:09:201
579110993323,7cyclictest639640-21ssh09:45:070
579112993225,4cyclictest957-21mta-sts-daemon10:39:252
579112993225,4cyclictest957-21mta-sts-daemon10:39:252
579112993225,4cyclictest687814-21wc10:55:022
579112993225,4cyclictest622646-21kworker/u8:1+xprtiod10:25:002
579112993225,4cyclictest622646-21kworker/u8:1+xprtiod10:25:002
579112993225,4cyclictest591095-21munin-plugin-st07:50:002
579112993225,4cyclictest581968-21unixbench-2d07:19:282
579112993224,5cyclictest583190-21sendmail_mailqu07:24:252
579112993223,5cyclictest585640-21fschecks_time07:34:092
57911299322,23cyclictest683644-21ssh10:49:232
57911299322,22cyclictest642084-21ps09:49:292
57911199325,5cyclictest171rcu_preempt12:39:231
57911199325,5cyclictest171rcu_preempt12:39:231
57911199322,6cyclictest171rcu_preempt10:59:171
579111993224,5cyclictest959-21mta-sts-daemon09:40:431
57911199322,23cyclictest680212-21ssh10:44:251
579111993221,6cyclictest0-21swapper/109:34:291
579110993221,8cyclictest583002-21ntp_kernel_pll_07:24:180
579112993124,3cyclictest0-21swapper/211:04:172
579112993122,6cyclictest421-21avahi-daemon07:29:052
57911299311,24cyclictest753828-21ssh12:29:182
57911199314,21cyclictest703973-21ssh11:19:171
579111993124,4cyclictest596984-21unixbench_multi08:09:451
579111993124,4cyclictest0-21swapper/112:15:011
579112993024,3cyclictest0-21swapper/209:09:182
579112993023,4cyclictest955-21mta-sts-daemon08:09:222
579112993023,3cyclictest0-21swapper/211:44:222
579112993023,3cyclictest0-21swapper/207:10:022
579111993023,4cyclictest0-21swapper/110:09:221
579111993021,6cyclictest604591-21systemd-run08:35:011
579111993019,7cyclictest613596-21systemd-run09:05:011
579111993015,10cyclictest620594-21sshd09:18:121
579112992921,5cyclictest655499-21latency10:09:142
579112992921,5cyclictest646054-21latency_hist09:55:022
579111992924,2cyclictest0-21swapper/111:30:011
579111992921,6cyclictest592619-21latency_hist07:55:021
579111992921,5cyclictest618825-21unixbench_singl09:14:291
579111992921,3cyclictest0-21swapper/110:54:111
57911299282,18cyclictest3987647-21cupsd08:59:192
579112992820,6cyclictest627706-21/usr/sbin/munin09:29:142
579112992820,5cyclictest610524-21grep08:54:352
579112992819,6cyclictest613569-21cron09:05:012
579111992821,4cyclictest609075-21munin-plugin-st08:50:011
579111992818,6cyclictest0-21swapper/108:39:161
579112992720,4cyclictest955-21mta-sts-daemon09:20:112
579112992719,5cyclictest603614-21fschecks_count08:34:092
579111992721,3cyclictest0-21swapper/107:44:301
579111992721,3cyclictest0-21swapper/107:44:301
579111992720,5cyclictest594603-21kworker/u8:3+xprtiod08:05:011
579110992717,7cyclictest579122-21munin-run07:10:020
579112992619,4cyclictest594631-21fschecks_count08:04:112
579112992617,5cyclictest600467-21aten_r2power_po08:24:082
57911199262,18cyclictest659212-21ntpq10:14:171
579111992620,3cyclictest0-21swapper/107:30:011
579111992619,4cyclictest599-21lldpd07:37:231
579111992618,5cyclictest421-21avahi-daemon08:22:511
57911199253,16cyclictest609982-21sshd08:54:241
579111992519,4cyclictest956-21mta-sts-daemon08:29:151
579111992519,4cyclictest956-21mta-sts-daemon08:29:151
579111992517,5cyclictest580268-21switchtime07:14:261
579111992515,6cyclictest579116-21true07:10:021
579111992513,7cyclictest266-21systemd-journal07:50:001
57911199251,18cyclictest607013-21sshd08:44:201
579112992417,4cyclictest595613-21kworker/u8:9-xprtiod08:15:012
57911199242,17cyclictest600077-21munin-run08:20:011
579111992418,3cyclictest0-21swapper/107:24:091
579111992217,3cyclictest0-21swapper/107:34:271
57911299201,14cyclictest592047-21ntpq07:54:202
57911399173,8cyclictest0-21swapper/310:14:173
57911399172,8cyclictest0-21swapper/309:54:223
57911399162,7cyclictest0-21swapper/312:14:203
57911399162,7cyclictest0-21swapper/311:44:183
57911399162,6cyclictest0-21swapper/311:24:163
57911399152,7cyclictest0-21swapper/312:04:203
57911399152,7cyclictest0-21swapper/311:57:173
57911399152,6cyclictest0-21swapper/312:19:243
57911399151,8cyclictest0-21swapper/309:49:283
57911399151,7cyclictest0-21swapper/308:29:203
57911399151,7cyclictest0-21swapper/308:29:203
57911399144,6cyclictest0-21swapper/312:29:193
57911399144,5cyclictest0-21swapper/311:14:223
57911399143,7cyclictest0-21swapper/310:49:203
57911399143,6cyclictest0-21swapper/312:39:233
57911399143,6cyclictest0-21swapper/312:39:233
57911399143,6cyclictest0-21swapper/310:04:153
57911399142,7cyclictest0-21swapper/310:59:183
57911399142,7cyclictest0-21swapper/310:39:213
57911399142,7cyclictest0-21swapper/310:39:213
57911399142,6cyclictest0-21swapper/311:04:193
57911399141,8cyclictest0-21swapper/310:19:183
57911399141,8cyclictest0-21swapper/310:19:183
57911399141,8cyclictest0-21swapper/309:29:203
57911399141,7cyclictest0-21swapper/312:34:163
57911399141,7cyclictest0-21swapper/312:34:163
57911399141,7cyclictest0-21swapper/311:54:223
57911399141,7cyclictest0-21swapper/311:19:213
57911399141,7cyclictest0-21swapper/307:14:203
57911399141,6cyclictest0-21swapper/308:39:183
57911399141,6cyclictest0-21swapper/307:49:203
57911399133,5cyclictest0-21swapper/311:09:173
57911399133,5cyclictest0-21swapper/309:14:153
57911399132,6cyclictest0-21swapper/311:29:213
57911399132,6cyclictest0-21swapper/309:19:173
57911399132,6cyclictest0-21swapper/307:24:213
57911399132,4cyclictest0-21swapper/307:29:153
57911399131,8cyclictest0-21swapper/309:24:103
57911399131,7cyclictest0-21swapper/310:54:173
57911399131,7cyclictest0-21swapper/310:44:213
57911399131,7cyclictest0-21swapper/310:09:223
57911399131,7cyclictest0-21swapper/309:59:183
57911399131,6cyclictest0-21swapper/312:09:173
57911399131,6cyclictest0-21swapper/308:49:183
57911399131,6cyclictest0-21swapper/308:34:173
57911399131,6cyclictest0-21swapper/307:19:203
57911399130,7cyclictest0-21swapper/312:24:193
57911399123,7cyclictest0-21swapper/308:54:153
57911399123,6cyclictest0-21swapper/311:49:113
57911399123,5cyclictest0-21swapper/309:39:203
57911399122,6cyclictest0-21swapper/308:09:193
57911399121,7cyclictest0-21swapper/310:29:143
57911399121,7cyclictest0-21swapper/310:29:143
57911399121,7cyclictest0-21swapper/309:34:253
57911399121,6cyclictest0-21swapper/309:44:113
57911399121,6cyclictest0-21swapper/309:10:033
57911399121,6cyclictest0-21swapper/309:04:173
57911399121,6cyclictest0-21swapper/308:19:173
57911399121,6cyclictest0-21swapper/308:04:223
57911399121,6cyclictest0-21swapper/307:54:233
57911399121,5cyclictest0-21swapper/311:32:143
57911399114,4cyclictest0-21swapper/308:44:283
57911399112,5cyclictest0-21swapper/308:24:213
57911399111,6cyclictest0-21swapper/310:30:133
57911399111,6cyclictest0-21swapper/310:30:133
57911399111,6cyclictest0-21swapper/310:24:083
57911399111,6cyclictest0-21swapper/310:24:083
57911399111,5cyclictest0-21swapper/307:39:203
57911399110,6cyclictest0-21swapper/308:14:263
57911399103,5cyclictest0-21swapper/311:39:243
57911399101,5cyclictest0-21swapper/307:34:093
57911399100,5cyclictest0-21swapper/308:59:163
5791139991,5cyclictest0-21swapper/307:44:083
5791139991,5cyclictest0-21swapper/307:44:083
5791139991,4cyclictest0-21swapper/307:55:253
5791139971,3cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional