You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-01 - 20:45

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Mon Dec 01, 2025 12:39:41)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
456149995852,4cyclictest0-21swapper/111:24:191
456149995852,4cyclictest0-21swapper/111:24:191
456150995727,7cyclictest3827321-21cups-browsed10:02:042
456150995629,6cyclictest266-21systemd-journal11:10:012
456150995627,4cyclictest0-21swapper/210:49:252
456149995640,10cyclictest602879-21ssh11:57:481
456149995143,4cyclictest0-21swapper/110:19:551
456149995143,4cyclictest0-21swapper/110:19:551
456149995040,7cyclictest1-21systemd10:00:011
456149995040,7cyclictest1-21systemd10:00:011
456149995040,6cyclictest577945-21cut11:20:001
456149995040,6cyclictest577945-21cut11:20:001
45614899496,14cyclictest14-21ksoftirqd/009:49:240
456150994840,3cyclictest0-21swapper/209:34:172
456149994738,6cyclictest498598-21systemd-run09:20:011
456148994639,4cyclictest422-21dbus-daemon08:20:010
456148994632,11cyclictest614560-21wc12:14:240
456149994536,6cyclictest490703-21systemd-run09:05:021
456148994533,5cyclictest0-21swapper/011:44:190
456148994524,14cyclictest12450irq/32-eth010:54:180
45614899451,41cyclictest1-21systemd12:40:010
45614899451,41cyclictest1-21systemd12:40:010
456150994436,5cyclictest422-21dbus-daemon10:34:272
456149994438,3cyclictest0-21swapper/107:15:011
456149994436,4cyclictest0-21swapper/108:04:171
456148994425,7cyclictest474-21in:imuxsock09:57:540
456148994425,7cyclictest474-21in:imuxsock09:57:540
456150994335,5cyclictest524753-21systemd-run10:00:022
456150994335,5cyclictest524753-21systemd-run10:00:022
456149994335,5cyclictest611675-21idleruntime-cro12:10:011
456148994337,4cyclictest478674-21latency_hist08:25:020
456148994332,7cyclictest3827321-21cups-browsed10:38:560
456148994324,8cyclictest12450irq/32-eth009:54:180
456150994231,8cyclictest633812-21/usr/sbin/munin12:40:012
456150994231,8cyclictest633812-21/usr/sbin/munin12:40:012
45614899422,35cyclictest618053-21ssh12:19:170
45614899422,35cyclictest618053-21ssh12:19:170
456148994222,17cyclictest487651-21latency_hist08:55:020
456150994132,6cyclictest487610-21cron08:55:012
456150994131,6cyclictest595154-21sh11:45:012
45614899415,30cyclictest554236-21ssh10:44:290
45614899413,30cyclictest467570-21sshd07:49:290
456150994033,4cyclictest0-21swapper/211:20:012
456150994033,4cyclictest0-21swapper/211:20:012
456150994032,5cyclictest489160-21munin-run09:00:012
45615099402,30cyclictest580648-21ssh11:24:242
45615099402,30cyclictest580648-21ssh11:24:242
456149994032,5cyclictest492201-21systemd-run09:10:011
456149994030,6cyclictest574389-21head11:14:301
456148994032,5cyclictest504028-21ntp_kernel_err09:29:220
456148994031,5cyclictest151ktimers/008:44:210
456150993933,3cyclictest0-21swapper/211:57:492
456150993931,5cyclictest588182-21cron11:35:012
456150993931,5cyclictest513353-21sort09:44:112
456150993929,7cyclictest611663-21cron12:10:012
456149993931,5cyclictest601636-21sh11:55:001
456149993930,6cyclictest1-21systemd09:55:011
45614899391,31cyclictest507203-21sh09:34:230
456149993833,2cyclictest0-21swapper/109:35:011
456149993831,4cyclictest0-21swapper/111:30:011
456149993830,5cyclictest489173-21plymouth09:00:011
456149993829,6cyclictest495743-21cron09:15:011
456149993829,4cyclictest0-21swapper/109:49:271
456149993824,8cyclictest492177-21diskmemload11:34:181
45614899389,19cyclictest437-21mta-sts-daemon07:29:190
456148993827,8cyclictest513352-21cut09:44:110
456148993825,10cyclictest573168-21ssh11:14:080
456148993823,11cyclictest266-21systemd-journal10:57:060
456150993731,3cyclictest0-21swapper/211:39:242
456150993729,5cyclictest574620-21cron11:15:002
456150993728,6cyclictest495555-21unixbench_singl09:14:292
456149993727,7cyclictest0-21swapper/112:05:031
456149993727,6cyclictest0-21swapper/109:39:221
456148993729,3cyclictest0-21swapper/012:35:010
456148993729,3cyclictest0-21swapper/012:35:010
456148993728,6cyclictest421-21avahi-daemon12:25:510
456148993728,6cyclictest421-21avahi-daemon12:25:510
45614899372,27cyclictest584010-21ssh11:29:230
456150993630,3cyclictest0-21swapper/209:40:012
456150993628,5cyclictest955-21mta-sts-daemon11:29:232
456150993628,5cyclictest537064-21needreboot10:19:162
456150993628,5cyclictest537064-21needreboot10:19:162
456150993628,5cyclictest468154-21systemd-run07:50:012
456150993627,6cyclictest531443-21systemd-run10:10:012
456149993630,3cyclictest0-21swapper/111:10:011
456149993628,5cyclictest613974-21fschecks_time12:14:101
456149993628,5cyclictest561469-21unixbench_singl10:54:301
456149993628,5cyclictest478664-21systemd-run08:25:011
456149993628,5cyclictest459691-21fschecks_time07:24:101
456149993626,5cyclictest0-21swapper/109:24:221
456149993624,9cyclictest594079-21latency11:44:151
45614999362,28cyclictest504032-21ssh09:29:221
45614999361,26cyclictest629355-21ssh12:34:201
45614999361,26cyclictest629355-21ssh12:34:201
456148993628,5cyclictest548216-21systemd-run10:35:010
456148993623,9cyclictest12450irq/32-eth008:49:310
45614899362,12cyclictest14-21ksoftirqd/010:14:190
45615099355,10cyclictest492177-21diskmemload09:19:352
456150993526,6cyclictest421-21avahi-daemon10:59:382
456150993520,12cyclictest456147-21cyclictest12:29:252
456150993520,12cyclictest456147-21cyclictest12:29:252
456149993525,6cyclictest633532-21grep12:39:301
456149993525,6cyclictest633532-21grep12:39:301
456149993525,6cyclictest597402-21latency11:49:151
456149993525,6cyclictest597402-21latency11:49:151
456149993517,12cyclictest437-21mta-sts-daemon07:44:191
456148993529,4cyclictest1-21systemd10:20:010
456148993529,4cyclictest1-21systemd10:20:010
456148993529,3cyclictest0-21swapper/007:25:020
456148993526,7cyclictest456957-21needreboot07:14:150
456148993526,6cyclictest3827321-21cups-browsed12:03:510
456148993522,11cyclictest191rcuc/007:34:180
456148993518,12cyclictest12450irq/32-eth010:04:190
456148993515,4cyclictest171rcu_preempt07:44:180
45614899351,12cyclictest511186-21kworker/u8:4+rpciod11:34:220
456150993426,5cyclictest568147-21sh11:05:012
456150993426,5cyclictest568147-21sh11:05:012
456150993426,5cyclictest466657-21systemd-run07:45:012
456150993425,6cyclictest475914-21aten_r2power_cu08:19:072
456150993423,7cyclictest478558-21tail08:24:302
456150993422,9cyclictest601326-21timerwakeupswit11:54:332
456149993427,4cyclictest513375-21fschecks_time09:44:111
456149993426,5cyclictest547848-21ssh10:34:311
456149993425,6cyclictest554896-21awk10:45:011
456149993425,5cyclictest0-21swapper/112:29:081
456149993425,5cyclictest0-21swapper/112:29:081
456149993423,8cyclictest698-21cron10:15:001
456149993418,10cyclictest492177-21diskmemload10:39:181
45614899343,21cyclictest567260-21sh11:04:190
45614899343,21cyclictest567260-21sh11:04:190
456148993427,4cyclictest543404-21aten_r2power_vo10:29:090
456148993427,4cyclictest0-21swapper/011:24:290
456148993427,4cyclictest0-21swapper/011:24:290
456148993423,5cyclictest151ktimers/008:09:260
456148993422,7cyclictest0-21swapper/011:57:510
45614899342,25cyclictest494949-21ssh09:14:180
456148993416,3cyclictest171rcu_preempt08:39:190
456148993416,3cyclictest171rcu_preempt08:39:190
456150993326,3cyclictest0-21swapper/210:24:162
456150993324,4cyclictest0-21swapper/212:24:232
456150993324,4cyclictest0-21swapper/212:24:232
456150993323,7cyclictest630165-21sh12:35:002
456150993323,7cyclictest630165-21sh12:35:002
456149993326,4cyclictest955-21mta-sts-daemon11:04:231
456149993326,4cyclictest955-21mta-sts-daemon11:04:231
456149993326,4cyclictest564902-21idleruntime-cro11:00:011
456149993325,5cyclictest539839-21aten_r2power_en10:24:071
456149993324,5cyclictest480148-21systemd-run08:30:011
456149993323,6cyclictest421-21avahi-daemon12:22:211
456149993323,6cyclictest421-21avahi-daemon12:22:211
456148993322,8cyclictest577314-21mailstats11:19:280
456148993322,8cyclictest577314-21mailstats11:19:280
45615099325,22cyclictest657-21snmpd07:29:192
456150993223,6cyclictest486158-21latency_hist08:50:022
45615099322,24cyclictest500771-21ssh09:24:162
456149993221,9cyclictest543877-21needreboot10:29:171
456148993225,4cyclictest475621-21munin-plugin-st08:15:010
456148993224,5cyclictest510818-21grep09:39:300
456148993218,4cyclictest171rcu_preempt11:39:210
456148993217,3cyclictest171rcu_preempt09:24:210
456148993211,14cyclictest171rcu_preempt11:49:190
456148993211,14cyclictest171rcu_preempt11:49:190
45615099313,21cyclictest560940-21ssh10:54:212
456150993124,3cyclictest0-21swapper/210:39:182
456150993123,5cyclictest618709-21unixbench_singl12:19:302
456150993123,5cyclictest618709-21unixbench_singl12:19:302
456150993121,6cyclictest492376-21latency_hist09:10:032
456150993120,8cyclictest520646-21ntp_offset09:54:232
45615099311,23cyclictest544042-21ssh10:29:222
456149993125,3cyclictest477139-21plymouth08:20:011
456149993125,3cyclictest0-21swapper/108:39:161
456149993125,3cyclictest0-21swapper/108:39:161
456149993124,4cyclictest3952105-21kworker/u8:5-xprtiod07:39:111
456149993123,5cyclictest558270-21munin-plugin-st10:50:011
456149993123,4cyclictest0-21swapper/111:40:011
456149993123,4cyclictest0-21swapper/107:19:201
45614899317,19cyclictest14-21ksoftirqd/010:09:230
456148993124,4cyclictest957-21mta-sts-daemon12:09:100
45614899312,14cyclictest492177-21diskmemload10:24:230
456148993119,9cyclictest0-21swapper/007:19:180
456148993117,4cyclictest171rcu_preempt09:04:180
456150993023,3cyclictest498147-21kworker/u8:16-xprtiod09:29:182
456150993022,4cyclictest465262-21kworker/u8:3-events_unbound07:57:292
456150993021,6cyclictest3827321-21cups-browsed08:44:262
456150993021,4cyclictest0-21swapper/212:04:192
456150993013,13cyclictest351ktimers/208:39:202
456150993013,13cyclictest351ktimers/208:39:202
456149993024,3cyclictest0-21swapper/110:10:021
456149993022,5cyclictest422-21dbus-daemon07:50:011
45614999302,24cyclictest527223-21ssh10:04:161
45614899302,24cyclictest570463-21ssh11:09:240
456148993020,3cyclictest472502-21sed08:04:280
456148993018,3cyclictest171rcu_preempt10:49:210
456148993015,4cyclictest171rcu_preempt07:54:180
45615099292,23cyclictest597689-21ssh11:49:242
45615099292,23cyclictest597689-21ssh11:49:242
456150992921,5cyclictest516514-21/usr/sbin/munin09:49:092
456150992920,7cyclictest613645-21/usr/sbin/munin12:14:272
456150992920,5cyclictest553975-21needreboot10:44:192
456150992919,8cyclictest472639-21latency_hist08:05:012
456149992923,3cyclictest0-21swapper/112:19:261
456149992923,3cyclictest0-21swapper/112:19:261
456149992922,4cyclictest959-21mta-sts-daemon08:09:221
45614899296,6cyclictest14-21ksoftirqd/008:59:220
456148992921,4cyclictest0-21swapper/009:16:440
456148992914,12cyclictest480664-21fschecks_count08:34:100
456148992914,11cyclictest621168-21aten2.4-expect12:24:060
456148992914,11cyclictest621168-21aten2.4-expect12:24:060
456148992910,2cyclictest171rcu_preempt07:39:190
456150992820,5cyclictest480411-21aten_r2power_cu08:34:062
456150992819,6cyclictest498-21wpa_supplicant10:13:392
456150992818,7cyclictest955-21mta-sts-daemon07:34:002
456149992820,5cyclictest959-21mta-sts-daemon07:27:461
456149992820,5cyclictest475532-21unixbench_multi08:14:341
45614999281,4cyclictest171rcu_preempt07:34:181
456150992718,7cyclictest456948-21needreboot07:14:152
456149992719,5cyclictest422-21dbus-daemon08:00:021
456148992720,4cyclictest492177-21sh09:10:010
456148992717,3cyclictest171rcu_preempt07:59:190
456148992711,3cyclictest171rcu_preempt11:54:110
456150992619,4cyclictest458583-21gcc07:19:192
456150992612,7cyclictest957-21mta-sts-daemon07:39:242
456149992621,2cyclictest0-21swapper/108:55:011
456149992619,4cyclictest456169-21latency_hist07:10:011
456150992520,3cyclictest956-21mta-sts-daemon07:51:382
456150992519,3cyclictest0-21swapper/208:14:172
45614999254,12cyclictest437-21mta-sts-daemon08:34:181
456149992518,4cyclictest959-21mta-sts-daemon07:50:091
456149992516,6cyclictest485300-21kernelversion08:49:201
456150992418,3cyclictest959-21mta-sts-daemon07:24:122
456149992416,5cyclictest484224-21systemctl08:44:261
456148992410,9cyclictest479063-21/usr/sbin/munin08:29:080
456150992314,6cyclictest490375-21switchtime09:04:272
456150992314,6cyclictest473001-21cut08:09:112
456150992217,2cyclictest0-21swapper/207:10:012
45615099212,14cyclictest479498-21ntpq08:29:162
45614899218,3cyclictest171rcu_preempt07:10:010
45615199183,8cyclictest0-21swapper/311:09:253
45615199183,8cyclictest0-21swapper/310:04:173
45615199171,8cyclictest0-21swapper/312:09:223
45615199163,8cyclictest0-21swapper/311:57:493
45615199163,8cyclictest0-21swapper/309:59:183
45615199163,8cyclictest0-21swapper/309:59:183
45615199163,6cyclictest0-21swapper/312:34:183
45615199163,6cyclictest0-21swapper/312:34:183
45615199162,8cyclictest0-21swapper/312:19:253
45615199162,8cyclictest0-21swapper/312:19:253
45615199162,8cyclictest0-21swapper/309:14:203
45615199162,6cyclictest0-21swapper/308:49:273
45615199161,9cyclictest0-21swapper/312:14:213
45615199161,8cyclictest0-21swapper/309:24:163
45615199152,8cyclictest0-21swapper/310:14:163
45615199152,7cyclictest0-21swapper/312:04:223
45615199152,7cyclictest0-21swapper/310:34:193
45615199152,7cyclictest0-21swapper/308:04:163
45615199152,6cyclictest0-21swapper/311:52:063
45615199152,6cyclictest0-21swapper/308:29:153
45615199151,8cyclictest0-21swapper/312:24:203
45615199151,8cyclictest0-21swapper/312:24:203
45615199151,8cyclictest0-21swapper/310:59:203
45615199143,7cyclictest0-21swapper/310:44:283
45615199143,6cyclictest0-21swapper/309:54:183
45615199143,6cyclictest0-21swapper/307:34:233
45615199143,6cyclictest0-21swapper/307:15:023
45615199142,7cyclictest0-21swapper/312:29:233
45615199142,7cyclictest0-21swapper/312:29:233
45615199142,7cyclictest0-21swapper/311:44:173
45615199142,7cyclictest0-21swapper/310:49:083
45615199142,6cyclictest0-21swapper/309:49:073
45615199142,6cyclictest0-21swapper/309:09:233
45615199141,8cyclictest0-21swapper/311:34:173
45615199141,7cyclictest0-21swapper/311:14:243
45615199141,7cyclictest0-21swapper/309:04:153
45615199141,6cyclictest0-21swapper/308:54:193
45615199136,5cyclictest0-21swapper/310:29:183
45615199133,6cyclictest0-21swapper/311:04:193
45615199133,6cyclictest0-21swapper/311:04:193
45615199133,6cyclictest0-21swapper/310:09:233
45615199133,4cyclictest0-21swapper/308:19:243
45615199132,7cyclictest0-21swapper/312:39:113
45615199132,7cyclictest0-21swapper/312:39:113
45615199132,7cyclictest0-21swapper/309:44:193
45615199132,6cyclictest0-21swapper/309:19:343
45615199132,6cyclictest0-21swapper/307:50:013
45615199132,5cyclictest0-21swapper/308:59:023
45615199132,5cyclictest0-21swapper/308:24:223
45615199132,5cyclictest0-21swapper/307:29:193
45615199131,8cyclictest0-21swapper/309:34:243
45615199131,7cyclictest0-21swapper/311:35:133
45615199131,7cyclictest0-21swapper/311:29:133
45615199131,7cyclictest0-21swapper/309:29:263
45615199131,6cyclictest0-21swapper/308:44:183
45615199131,6cyclictest0-21swapper/307:54:203
45615199123,6cyclictest0-21swapper/310:24:223
45615199123,6cyclictest0-21swapper/308:09:253
45615199122,6cyclictest0-21swapper/311:45:063
45615199122,6cyclictest0-21swapper/311:45:063
45615199122,6cyclictest0-21swapper/311:15:293
45615199122,6cyclictest0-21swapper/311:15:293
45615199122,5cyclictest0-21swapper/309:38:533
45615199121,7cyclictest0-21swapper/311:24:143
45615199121,7cyclictest0-21swapper/311:24:143
45615199121,7cyclictest0-21swapper/310:39:193
45615199121,7cyclictest0-21swapper/307:59:193
45615199121,6cyclictest0-21swapper/307:24:153
45615199121,6cyclictest0-21swapper/307:19:153
45615199120,6cyclictest0-21swapper/307:39:173
45615199112,6cyclictest0-21swapper/310:19:113
45615199112,6cyclictest0-21swapper/310:19:113
45615199112,5cyclictest0-21swapper/307:44:173
45615199111,6cyclictest0-21swapper/308:14:223
45615199102,6cyclictest0-21swapper/308:35:383
45615199102,6cyclictest0-21swapper/308:35:383
45615199102,6cyclictest0-21swapper/307:10:013
45615199102,5cyclictest0-21swapper/310:53:063
45615199102,4cyclictest0-21swapper/308:34:143
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional