You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-22 - 14:12

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Sat Nov 22, 2025 12:39:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3304217996318,35cyclictest3451660-21ssh11:59:210
3304217995850,4cyclictest0-21swapper/009:14:180
330421999553,38cyclictest3464725-21ssh12:19:152
3304217995438,13cyclictest3432221-21latency_hist11:30:010
3304217995438,13cyclictest3432221-21latency_hist11:30:010
3304217994944,3cyclictest0-21swapper/011:24:160
3304217994944,3cyclictest0-21swapper/011:24:160
3304217994842,3cyclictest0-21swapper/012:19:170
3304217994834,10cyclictest3378384-21ssh10:09:190
3304218994638,5cyclictest3385671-21sendmail10:20:011
3304218994638,5cyclictest3372378-21munin-run10:00:011
3304218994638,5cyclictest3372378-21munin-run10:00:011
3304217994640,4cyclictest12450irq/32-eth009:36:500
3304217994631,12cyclictest3412529-21systemd-run11:00:010
3304217994630,13cyclictest3338799-21latency_hist09:05:020
3304217994532,10cyclictest3375881-21cron10:05:010
3304219994435,5cyclictest0-21swapper/209:20:012
3304219994434,7cyclictest0-21swapper/212:39:172
330421799441,25cyclictest3459773-21ssh12:12:270
3304218994338,3cyclictest0-21swapper/109:49:191
3304218994334,6cyclictest266-21systemd-journal11:40:001
3304218994334,6cyclictest266-21systemd-journal11:40:001
3304217994333,6cyclictest0-21swapper/011:39:180
3304217994333,6cyclictest0-21swapper/011:39:180
3304217994327,14cyclictest0-21swapper/012:40:010
3304217994324,16cyclictest3394990-21grep10:34:390
3304219994234,5cyclictest0-21swapper/209:40:012
3304219994231,7cyclictest3380834-21/usr/sbin/munin10:14:372
3304218994234,5cyclictest3347375-21munin-run09:20:011
3304218994233,5cyclictest1-21systemd07:30:011
3304218994233,5cyclictest1-21systemd07:30:011
330421799421,24cyclictest3476614-21sh12:34:420
3304219994133,5cyclictest3313146-21head07:39:262
3304219994133,5cyclictest3313146-21head07:39:262
3304219994132,5cyclictest3304239-21latency_hist07:10:022
3304219994116,16cyclictest422-21dbus-daemon10:49:172
3304218994133,5cyclictest3317698-21grep07:55:011
3304218994133,5cyclictest266-21systemd-journal09:45:011
3304217994133,5cyclictest421-21avahi-daemon11:15:360
3304217994133,5cyclictest421-21avahi-daemon11:15:360
3304217994124,14cyclictest3320632-21unixbench_multi08:04:240
330421799412,21cyclictest3346516-21ntpq09:19:170
3304219994032,5cyclictest3325205-21munin-run08:20:012
330421999402,31cyclictest3371494-21ssh09:59:152
330421999402,31cyclictest3371494-21ssh09:59:152
3304219994023,11cyclictest421-21avahi-daemon11:59:252
3304219994017,15cyclictest959-21mta-sts-daemon10:39:132
3304218994032,5cyclictest422-21dbus-daemon07:34:181
3304218994032,5cyclictest3469443-21systemd-run12:25:021
3304218994031,5cyclictest3368902-21cron09:55:011
3304218994031,5cyclictest3368902-21cron09:55:011
3304217994032,5cyclictest3329770-21systemd-run08:35:010
3304217994032,5cyclictest1-21systemd09:55:020
3304217994032,5cyclictest1-21systemd09:55:020
3304217994024,5cyclictest3311727-21systemd07:35:020
3304217994023,14cyclictest3455778-21latency_hist12:05:020
3304217994022,15cyclictest0-21swapper/010:15:010
330421799402,19cyclictest3408262-21ntpq10:54:150
330421799401,36cyclictest3387737-21ssh10:24:080
330421799401,28cyclictest3417923-21ssh11:09:060
3304219993930,6cyclictest3431196-21grep11:29:102
3304219993930,6cyclictest3431196-21grep11:29:102
3304219993930,5cyclictest3445341-21unixbench_multi11:49:292
3304219993930,5cyclictest3445341-21unixbench_multi11:49:292
3304219993930,5cyclictest3331252-21idleruntime-cro08:40:012
3304218993932,3cyclictest0-21swapper/111:34:181
3304218993932,3cyclictest0-21swapper/111:34:181
3304218993931,5cyclictest3377882-21fschecks_time10:09:081
3304218993931,5cyclictest3356711-21latency_hist09:35:021
3304217993933,4cyclictest151ktimers/007:14:190
3304217993933,3cyclictest0-21swapper/009:44:270
3304217993932,3cyclictest0-21swapper/007:29:100
3304217993932,3cyclictest0-21swapper/007:29:100
3304217993925,9cyclictest12450irq/32-eth010:19:120
330421799392,19cyclictest3356047-21ntpq09:34:170
330421799391,26cyclictest3415000-21ssh11:04:140
330421799391,24cyclictest3470804-21ssh12:27:370
330421799391,21cyclictest3322709-21df_inode08:14:060
330421999384,27cyclictest3365391-21ssh09:49:162
3304219993830,5cyclictest3402332-21latency_hist10:45:012
3304218993830,5cyclictest2465701-21cups-browsed12:30:181
3304217993831,4cyclictest3352368-21ssh09:29:040
3304217993826,3cyclictest0-21swapper/010:44:210
3304217993822,12cyclictest0-21swapper/010:49:180
330421799381,33cyclictest3447042-21ssh11:52:460
330421799381,32cyclictest3326961-21apt-get08:26:320
330421799381,25cyclictest3398937-21latency_hist10:40:020
330421799380,19cyclictest3340259-21diskmemload11:32:000
330421799380,19cyclictest3340259-21diskmemload11:32:000
3304219993731,3cyclictest0-21swapper/212:25:022
3304219993730,4cyclictest3428891-21idleruntime-cro11:25:002
3304219993730,4cyclictest3428891-21idleruntime-cro11:25:002
3304218993731,3cyclictest0-21swapper/112:39:141
3304218993728,6cyclictest2465701-21cups-browsed12:18:111
3304217993729,5cyclictest1-21systemd07:20:020
3304217993725,9cyclictest0-21swapper/007:40:010
3304217993725,9cyclictest0-21swapper/007:40:010
330421799372,31cyclictest3365421-21ntpq09:49:170
330421799371,31cyclictest0-21swapper/008:49:100
330421799371,18cyclictest3340259-21diskmemload09:21:500
330421999361,25cyclictest3411721-21ssh10:59:192
3304218993630,3cyclictest0-21swapper/107:20:011
3304218993629,4cyclictest3442201-21unixbench_multi11:44:261
3304218993629,4cyclictest3442201-21unixbench_multi11:44:261
3304218993616,7cyclictest657-21snmpd10:41:321
330421799363,29cyclictest3335150-21ntpq08:54:120
330421799362,29cyclictest3468501-21sh12:24:140
330421799362,29cyclictest3332188-21ntpq08:44:140
330421799362,27cyclictest3330631-21ntpq08:39:140
330421799361,29cyclictest3315643-21ntpq07:49:210
330421799361,23cyclictest3419597-21ssh11:10:510
3304219993527,5cyclictest3306251-21fschecks_time07:19:062
330421999352,24cyclictest3461203-21ssh12:14:172
3304219993516,12cyclictest3409095-21latency_hist10:55:032
3304218993529,3cyclictest0-21swapper/111:54:031
3304218993529,3cyclictest0-21swapper/109:10:011
3304218993527,3cyclictest0-21swapper/112:04:141
330421899352,29cyclictest3424978-21ssh11:19:221
330421899352,29cyclictest3424978-21ssh11:19:221
3304217993528,4cyclictest3441183-21sh11:44:060
3304217993528,4cyclictest3441183-21sh11:44:060
3304217993527,5cyclictest2465678-21cupsd09:58:530
3304217993527,5cyclictest2465678-21cupsd09:58:530
330421799351,30cyclictest3443960-21ssh11:48:550
330421799351,30cyclictest3443960-21ssh11:48:550
330421799351,16cyclictest3316427-21kworker/u8:13+rpciod07:54:200
330421799350,32cyclictest3324221-21/usr/sbin/munin08:19:050
3304219993426,5cyclictest3388509-21grep10:24:242
3304219993425,6cyclictest3340278-21systemd-run09:10:012
3304219993425,5cyclictest0-21swapper/211:40:002
3304219993425,5cyclictest0-21swapper/211:40:002
3304219993424,6cyclictest3332776-21systemd-run08:45:022
3304219993423,8cyclictest3394869-21sshd10:34:382
3304219993416,3cyclictest0-21swapper/209:00:012
3304218993427,4cyclictest0-21swapper/111:00:011
3304218993426,5cyclictest3334802-21fschecks_time08:54:051
3304218993425,5cyclictest3333177-21aten_r2power_vo08:49:031
3304218993417,3cyclictest171rcu_preempt10:14:271
3304218993417,3cyclictest171rcu_preempt09:24:171
330421899341,27cyclictest3418483-21ssh11:09:181
330421799342,28cyclictest3321656-21ntpq08:09:160
3304217993414,15cyclictest3391316-21ssh10:29:170
330421799341,30cyclictest3458011-21ntpq12:09:200
330421799341,29cyclictest3325928-21latency08:24:100
330421799341,29cyclictest3308218-21ntpq07:24:220
330421799341,29cyclictest3308218-21ntpq07:24:220
330421799341,28cyclictest3339703-21ntpq09:09:310
330421799341,28cyclictest3318574-21ntpq07:59:120
3304219993327,3cyclictest0-21swapper/209:54:252
3304219993327,3cyclictest0-21swapper/209:54:252
3304219993327,3cyclictest0-21swapper/208:00:022
3304219993325,5cyclictest3442268-21head11:44:262
3304219993325,5cyclictest3442268-21head11:44:262
3304219993325,5cyclictest0-21swapper/211:14:152
3304219993323,6cyclictest3473148-21systemd-run12:30:012
330421999332,26cyclictest3384738-21ssh10:19:132
330421899339,6cyclictest0-21swapper/108:39:151
3304218993325,5cyclictest957-21mta-sts-daemon12:11:191
3304218993325,5cyclictest3332752-21cron08:45:011
3304218993324,6cyclictest0-21swapper/110:35:011
3304218993318,3cyclictest171rcu_preempt11:14:171
330421899331,28cyclictest3408183-21ssh10:54:111
330421799333,27cyclictest3336607-21ntpq08:59:110
330421799332,27cyclictest3314076-21ntpq07:44:120
3304219993226,3cyclictest0-21swapper/211:34:252
3304219993226,3cyclictest0-21swapper/211:34:252
3304219993225,4cyclictest3359580-21kworker/u8:10-xprtiod09:44:272
3304219993225,4cyclictest266-21systemd-journal08:35:002
3304219993224,4cyclictest3325765-21fschecks_time08:24:082
3304219993224,4cyclictest0-21swapper/208:30:012
3304219993223,5cyclictest3375278-21/usr/sbin/munin10:04:232
3304219993223,5cyclictest1-21systemd11:20:022
3304219993223,5cyclictest1-21systemd11:20:022
330421999321,26cyclictest335607250ssh09:34:182
330421999320,27cyclictest3414971-21ssh11:04:132
330421899329,3cyclictest171rcu_preempt08:29:171
3304218993226,4cyclictest3304216-21cyclictest08:35:011
3304218993224,5cyclictest3312020-21aten_r2power_en07:39:031
3304218993224,5cyclictest3312020-21aten_r2power_en07:39:031
3304218993224,4cyclictest3334492-21kworker/u8:2-xprtiod09:00:011
3304218993223,6cyclictest3359419-21unixbench-2d09:39:271
3304218993223,6cyclictest266-21systemd-journal11:24:311
3304218993223,6cyclictest266-21systemd-journal11:24:311
3304218993221,8cyclictest3445324-21unixbench_multi11:49:291
3304218993221,8cyclictest3445324-21unixbench_multi11:49:291
3304218993219,3cyclictest171rcu_preempt09:14:211
3304218993216,6cyclictest3373301-21sh10:01:411
3304218993214,5cyclictest0-21swapper/111:59:191
330421999313,23cyclictest3475955-21ssh12:34:202
3304219993123,5cyclictest3392080-21cron10:30:012
3304219993121,7cyclictest3338755-21cron09:05:012
3304219993120,8cyclictest3311034-21needreboot07:34:132
330421899319,4cyclictest0-21swapper/107:59:111
3304218993125,3cyclictest0-21swapper/110:40:011
3304218993122,6cyclictest3458570-21vmstat12:09:311
3304218993122,5cyclictest956-21mta-sts-daemon12:29:131
3304219993021,5cyclictest0-21swapper/208:04:162
330421899302,3cyclictest171rcu_preempt08:19:121
3304218993016,2cyclictest171rcu_preempt07:24:201
3304218993016,2cyclictest171rcu_preempt07:24:201
3304218993014,8cyclictest957-21mta-sts-daemon09:26:391
3304218993014,3cyclictest0-21swapper/108:09:191
3304218993014,2cyclictest171rcu_preempt10:46:031
3304218993011,4cyclictest171rcu_preempt11:27:151
3304218993011,4cyclictest171rcu_preempt11:27:151
3304218993011,4cyclictest171rcu_preempt10:25:061
3304218993010,4cyclictest0-21swapper/111:04:151
3304217993017,3cyclictest0-21swapper/007:10:020
3304219992922,4cyclictest956-21mta-sts-daemon11:09:242
3304219992921,5cyclictest3305695-21sh07:15:012
3304219992920,6cyclictest421-21avahi-daemon09:29:272
3304219992912,6cyclictest421-21avahi-daemon10:06:002
3304218992922,5cyclictest2465701-21cups-browsed07:44:181
3304218992921,5cyclictest3323714-21systemd-run08:15:011
3304218992917,3cyclictest171rcu_preempt09:04:131
3304218992915,3cyclictest171rcu_preempt08:04:131
330421999287,17cyclictest959-21mta-sts-daemon08:14:162
330421999282,21cyclictest3457947-21ssh12:09:172
330421999282,19cyclictest3343215-21ssh09:14:192
3304219992820,5cyclictest3316740-21fschecks_count07:54:112
3304218992815,3cyclictest171rcu_preempt08:24:131
3304218992813,3cyclictest171rcu_preempt10:24:121
3304219992719,5cyclictest3331277-21kworker/u8:1+xprtiod08:55:012
3304219992718,6cyclictest266-21systemd-journal09:20:042
3304219992620,3cyclictest3308811-21kworker/u8:3-xprtiod07:49:302
3304219992620,3cyclictest0-21swapper/211:54:242
3304218992621,2cyclictest0-21swapper/107:14:071
3304218992619,4cyclictest3301503-21kworker/u8:4-xprtiod07:50:021
330422099258,8cyclictest0-21swapper/311:19:233
330422099258,8cyclictest0-21swapper/311:19:233
3304219992519,3cyclictest351ktimers/208:49:172
3304219992518,4cyclictest3322148-21unixbench_singl08:09:262
3304219992518,3cyclictest0-21swapper/212:04:062
3304219992418,3cyclictest0-21swapper/207:25:012
3304219992418,3cyclictest0-21swapper/207:25:012
3304218992418,3cyclictest261rcuc/107:10:021
330421999238,3cyclictest171rcu_preempt07:44:102
330422099228,7cyclictest0-21swapper/310:19:183
330422099224,10cyclictest0-21swapper/311:29:133
330422099224,10cyclictest0-21swapper/311:29:133
3304219992216,3cyclictest956-21mta-sts-daemon07:26:462
3304219992216,3cyclictest956-21mta-sts-daemon07:26:462
330422099218,7cyclictest0-21swapper/311:04:143
330422099217,8cyclictest0-21swapper/311:44:123
330422099217,8cyclictest0-21swapper/311:44:123
330422099217,7cyclictest0-21swapper/311:24:163
330422099217,7cyclictest0-21swapper/311:24:163
330422099208,7cyclictest0-21swapper/312:04:113
330422099208,6cyclictest0-21swapper/309:49:163
330422099207,8cyclictest0-21swapper/312:09:193
330422099207,8cyclictest0-21swapper/311:14:133
330422099207,7cyclictest0-21swapper/312:39:133
330422099207,7cyclictest0-21swapper/310:44:143
330422099207,7cyclictest0-21swapper/310:34:273
330422099199,6cyclictest0-21swapper/309:34:153
330422099199,6cyclictest0-21swapper/307:24:173
330422099199,6cyclictest0-21swapper/307:24:173
330422099199,5cyclictest0-21swapper/312:24:113
330422099198,7cyclictest0-21swapper/311:54:123
330422099198,7cyclictest0-21swapper/309:44:133
330422099198,7cyclictest0-21swapper/309:19:183
330422099197,7cyclictest0-21swapper/312:19:153
330422099197,7cyclictest0-21swapper/312:14:153
330422099197,7cyclictest0-21swapper/310:09:133
330422099197,6cyclictest0-21swapper/310:04:023
330422099197,6cyclictest0-21swapper/309:59:143
330422099197,6cyclictest0-21swapper/309:59:143
330422099196,8cyclictest0-21swapper/309:36:513
330422099193,11cyclictest0-21swapper/310:29:103
330422099191,14cyclictest0-21swapper/311:39:163
330422099191,14cyclictest0-21swapper/311:39:163
330422099189,5cyclictest0-21swapper/309:54:213
330422099189,5cyclictest0-21swapper/309:54:213
330422099188,6cyclictest0-21swapper/310:39:153
330422099187,7cyclictest0-21swapper/312:29:123
330422099187,7cyclictest0-21swapper/310:14:303
330422099187,7cyclictest0-21swapper/309:14:173
330422099186,7cyclictest0-21swapper/310:59:113
330422099186,6cyclictest0-21swapper/309:24:133
330422099181,12cyclictest0-21swapper/311:09:133
330422099181,12cyclictest0-21swapper/310:49:133
330422099178,6cyclictest0-21swapper/310:24:113
330422099178,5cyclictest0-21swapper/309:09:303
330422099177,6cyclictest0-21swapper/311:49:133
330422099177,6cyclictest0-21swapper/311:49:133
330422099177,6cyclictest0-21swapper/311:34:113
330422099177,6cyclictest0-21swapper/311:34:113
330422099177,6cyclictest0-21swapper/310:54:123
330422099177,5cyclictest0-21swapper/307:49:183
330422099176,7cyclictest0-21swapper/311:59:163
330422099176,7cyclictest0-21swapper/308:44:123
330422099173,10cyclictest0-21swapper/312:34:173
330422099171,11cyclictest0-21swapper/309:04:123
330422099171,11cyclictest0-21swapper/308:26:313
330422099171,11cyclictest0-21swapper/307:44:123
330422099171,11cyclictest0-21swapper/307:14:143
330422099168,5cyclictest0-21swapper/308:04:133
330422099167,5cyclictest0-21swapper/309:29:053
330422099167,5cyclictest0-21swapper/308:59:153
330422099167,5cyclictest0-21swapper/308:19:113
330422099167,5cyclictest0-21swapper/308:09:143
330422099167,5cyclictest0-21swapper/307:54:203
330422099167,5cyclictest0-21swapper/307:39:133
330422099167,5cyclictest0-21swapper/307:39:133
330422099166,7cyclictest0-21swapper/308:39:343
330422099161,11cyclictest0-21swapper/308:54:123
330422099161,11cyclictest0-21swapper/307:59:123
330422099161,11cyclictest0-21swapper/307:34:153
330422099160,13cyclictest0-21swapper/307:10:013
330422099158,5cyclictest0-21swapper/308:24:103
330422099158,5cyclictest0-21swapper/308:14:183
330422099157,5cyclictest0-21swapper/307:19:123
330422099157,4cyclictest0-21swapper/307:29:053
330422099157,4cyclictest0-21swapper/307:29:053
330422099151,10cyclictest0-21swapper/308:49:113
330422099151,10cyclictest0-21swapper/308:34:143
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional