You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-03 - 21:29

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot5s.osadl.org (updated Wed Dec 03, 2025 12:44:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
501938995750,5cyclictest597019-21kworker/u8:2+events_unbound12:10:171
501935995136,10cyclictest121ktimers/010:20:110
501941994734,8cyclictest518853-21grep07:55:252
501941994632,11cyclictest311ktimers/208:55:142
501946994433,8cyclictest391ktimers/311:25:113
501946994433,7cyclictest600321-21grep11:50:253
501938994434,7cyclictest581776-21sendmail11:00:001
501935994436,4cyclictest580049-21cron10:55:010
501941994335,4cyclictest566006-21munin-plugin-st10:15:012
50193599436,19cyclictest516300-21sort07:50:010
50193599436,19cyclictest516300-21sort07:50:000
501941994235,4cyclictest581765-21munin-plugin-st11:00:002
501941994233,6cyclictest527599-21grep08:20:272
50193599414,20cyclictest504550-21fschecks_count07:15:180
501935994134,4cyclictest547033-21munin-plugin-st09:20:010
501935994129,9cyclictest121ktimers/009:30:140
501946994028,8cyclictest391ktimers/312:35:143
501938994032,4cyclictest503087-21unixbench_multi07:10:261
50193899402,8cyclictest24-21ksoftirqd/110:00:131
501938993911,4cyclictest131rcu_preempt08:50:131
50193899390,9cyclictest24-21ksoftirqd/110:25:121
501946993831,4cyclictest574735-21sendmail10:40:003
501938993832,4cyclictest560826-21kworker/u8:1+events_unbound10:10:021
501938993825,9cyclictest509295-21idleruntime-cro07:30:011
501938993814,7cyclictest131rcu_preempt08:05:121
501941993725,10cyclictest311ktimers/207:35:162
501938993729,4cyclictest0-21swapper/111:20:001
501938993710,4cyclictest131rcu_preempt07:15:141
501938993630,4cyclictest531252-21kworker/u8:0+events_unbound09:01:421
501938993628,4cyclictest0-21swapper/110:30:171
501935993628,4cyclictest0-21swapper/008:55:290
501935993610,8cyclictest11-21ksoftirqd/011:15:020
501946993526,6cyclictest520941-21kworker/u8:0-ext4-rsv-conversion08:20:013
501941993527,4cyclictest0-21swapper/212:05:232
501938993525,6cyclictest0-21swapper/110:20:251
501938993514,7cyclictest131rcu_preempt08:25:131
50193599359,9cyclictest573265-21/usr/sbin/munin10:35:260
501935993528,4cyclictest512699-21latency_hist07:40:010
501935993526,4cyclictest0-21swapper/009:10:000
501941993426,5cyclictest590256-21kworker/u8:0+events_unbound11:30:112
501938993425,6cyclictest592212-21sh11:30:011
50193599349,6cyclictest11-21ksoftirqd/008:50:140
50193599348,22cyclictest526567-21latency_hist08:20:020
501946993325,4cyclictest0-21swapper/311:55:253
501946993325,4cyclictest0-21swapper/310:50:163
501946993324,6cyclictest512704-21sendmail07:40:013
501946993324,5cyclictest0-21swapper/307:20:003
501946993322,7cyclictest548741-21debian-sa109:25:013
501941993327,3cyclictest0-21swapper/210:35:002
50194199331,7cyclictest131rcu_preempt09:35:152
501938993326,5cyclictest560826-21kworker/u8:1+events_unbound10:15:031
501938993325,5cyclictest609286-21sendmail12:20:011
501938993325,4cyclictest0-21swapper/111:50:241
501938993324,5cyclictest0-21swapper/108:35:261
501938993322,7cyclictest592083-21kworker/u8:1+events_unbound11:40:251
501938993322,7cyclictest231ktimers/108:20:151
501938993322,7cyclictest0-21swapper/107:55:001
50193599339,6cyclictest11-21ksoftirqd/008:35:020
50193599336,9cyclictest518791-21mailstats07:55:240
501935993325,4cyclictest0-21swapper/011:00:260
501935993325,3cyclictest0-21swapper/011:35:010
501946993225,3cyclictest0-21swapper/312:10:273
501946993224,4cyclictest0-21swapper/312:20:153
501946993224,4cyclictest0-21swapper/311:10:163
501941993224,4cyclictest0-21swapper/212:25:182
501941993224,4cyclictest0-21swapper/212:00:252
501941993224,4cyclictest0-21swapper/212:00:242
501941993223,5cyclictest0-21swapper/209:40:262
501938993225,4cyclictest0-21swapper/109:30:281
501938993225,3cyclictest0-21swapper/111:10:011
501938993221,6cyclictest0-21swapper/109:40:011
501935993214,12cyclictest524174-21grep08:10:250
501946993123,4cyclictest0-21swapper/309:00:263
501946993122,7cyclictest391ktimers/308:25:133
501946993121,6cyclictest0-21swapper/309:35:013
501941993125,4cyclictest533037-21kworker/u8:1+events_unbound08:39:482
501941993123,4cyclictest0-21swapper/211:40:002
501935993123,4cyclictest0-21swapper/011:55:250
501946993023,4cyclictest559179-21auditctl09:55:013
501946993022,5cyclictest496017-21kworker/u8:2+events_unbound07:29:363
501946993022,4cyclictest0-21swapper/311:20:283
501946993022,4cyclictest0-21swapper/309:05:253
501946993022,4cyclictest0-21swapper/308:35:263
501946993020,7cyclictest538676-21cut08:55:173
501941993022,4cyclictest0-21swapper/211:50:242
501941993022,4cyclictest0-21swapper/210:45:172
50193899308,4cyclictest0-21swapper/108:34:001
50193899305,14cyclictest557622-21latency_hist09:50:011
501938993022,4cyclictest0-21swapper/109:15:161
501938993021,6cyclictest0-21swapper/110:00:001
501938993021,5cyclictest550038-21head09:25:251
501938993015,4cyclictest131rcu_preempt10:45:181
501935993023,4cyclictest517807-21cron07:55:000
501935993016,9cyclictest503851-21seq07:15:020
501935993016,11cyclictest603775-21/usr/sbin/munin12:00:250
501935993016,11cyclictest603775-21/usr/sbin/munin12:00:250
501935993016,11cyclictest506427-21/usr/sbin/munin07:20:180
501935993011,4cyclictest11-21ksoftirqd/009:04:510
501946992922,4cyclictest0-21swapper/310:20:253
501946992921,4cyclictest0-21swapper/309:40:253
501941992919,7cyclictest536582-21auditctl08:50:012
501941992916,8cyclictest524156-21grep08:10:262
50193899295,5cyclictest483547-21kworker/1:2-mm_percpu_wq09:25:011
501938992923,4cyclictest510761-21kworker/u8:0+events_unbound07:35:011
501938992921,4cyclictest0-21swapper/111:00:271
501938992915,10cyclictest231ktimers/112:20:131
50193599299,15cyclictest557477-21sh09:50:010
501935992921,4cyclictest588798-21munin-plugin-st11:20:010
501935992917,7cyclictest583721-21apt-get11:05:120
501946992821,4cyclictest0-21swapper/311:40:263
501946992821,4cyclictest0-21swapper/310:50:013
501946992820,5cyclictest585310-21sort11:10:013
501946992820,4cyclictest0-21swapper/311:30:253
501941992822,4cyclictest585053-21kworker/u8:2+events_unbound11:15:092
501941992822,3cyclictest578594-21/usr/sbin/munin10:50:192
501941992821,4cyclictest0-21swapper/211:20:282
501941992820,4cyclictest0-21swapper/211:05:242
501941992820,4cyclictest0-21swapper/207:35:012
501941992819,6cyclictest537864-21grep08:50:252
501941992818,6cyclictest507158-21grep07:20:272
501938992822,3cyclictest0-21swapper/108:45:171
501938992821,5cyclictest572737-21kworker/u8:1+events_unbound10:36:541
501938992821,4cyclictest0-21swapper/107:55:251
501938992821,3cyclictest0-21swapper/111:55:241
501938992819,6cyclictest0-21swapper/111:40:011
501938992819,4cyclictest0-21swapper/108:40:251
501938992817,7cyclictest590124-21grep11:20:281
50193599289,15cyclictest561677-21ntpq10:00:220
501935992820,4cyclictest0-21swapper/010:55:240
501935992817,6cyclictest0-21swapper/009:45:010
501935992816,9cyclictest563956-21cron10:06:000
501935992815,10cyclictest533017-21sshd08:38:240
501935992815,10cyclictest508574-21/usr/sbin/munin07:25:440
501935992813,10cyclictest590107-21grep11:20:270
501935992811,13cyclictest612269-21grep12:25:280
501935992811,12cyclictest576011-21unixbench_multi10:40:260
501935992810,4cyclictest604511-21df12:05:210
501946992721,3cyclictest0-21swapper/310:05:013
501946992719,4cyclictest0-21swapper/311:20:013
501946992719,4cyclictest0-21swapper/309:35:173
501946992718,5cyclictest567899-21grep10:20:013
501941992721,3cyclictest0-21swapper/209:20:242
501941992720,3cyclictest0-21swapper/207:20:012
501941992719,5cyclictest550127-21kworker/u8:2+events_unbound09:26:232
501941992719,4cyclictest0-21swapper/208:00:302
50193899277,3cyclictest0-21swapper/111:10:181
501938992721,3cyclictest0-21swapper/110:50:151
501938992720,5cyclictest519161-21kworker/u8:1+events_unbound08:10:201
501938992719,4cyclictest0-21swapper/107:25:011
50193899270,16cyclictest563820-21grep10:05:261
50193599279,15cyclictest594930-21/usr/sbin/munin11:35:240
50193599277,6cyclictest0-21swapper/008:25:200
50193599277,18cyclictest553829-21sshd09:37:230
50193599277,15cyclictest0-21swapper/008:00:150
501935992722,3cyclictest607793-21apt-get12:15:130
501935992719,6cyclictest0-21swapper/007:30:210
501935992718,6cyclictest0-21swapper/010:00:010
501935992717,7cyclictest595990-21/usr/sbin/munin11:40:160
501935992717,7cyclictest577759-21/usr/sbin/munin10:45:260
501935992717,7cyclictest533411-21munin-node08:40:100
501935992714,3cyclictest0-21swapper/010:31:130
501935992714,10cyclictest557982-21/usr/sbin/munin09:50:170
501935992710,12cyclictest0-21swapper/010:30:010
501946992618,4cyclictest0-21swapper/312:05:313
501946992617,4cyclictest0-21swapper/308:54:493
501946992614,4cyclictest503217-21kworker/u8:1+events_unbound07:14:213
501946992613,9cyclictest215-21dbus-daemon08:14:413
501941992619,4cyclictest546594-21grep09:15:252
501941992619,4cyclictest0-21swapper/212:35:252
501941992616,7cyclictest1-21systemd12:30:172
501941992615,8cyclictest531043-21grep08:30:262
50193899268,5cyclictest131rcu_preempt11:45:351
501938992619,5cyclictest533037-21kworker/u8:1+events_unbound09:11:581
501938992619,4cyclictest0-21swapper/108:15:251
501938992618,6cyclictest555643-21kworker/u8:0+events_unbound09:44:181
501938992618,6cyclictest231ktimers/109:50:171
501938992618,4cyclictest0-21swapper/108:59:591
501938992616,6cyclictest231ktimers/109:05:111
50193899261,12cyclictest0-21swapper/112:09:161
50193599269,14cyclictest606268-21fschecks_count12:10:180
50193599268,5cyclictest0-21swapper/009:20:230
50193599268,14cyclictest528060-21sh08:25:010
50193599263,20cyclictest566086-21latency_hist10:15:010
501935992619,3cyclictest610084-21ntp_states12:20:200
501935992617,7cyclictest615073-21meminfo12:35:200
501935992617,6cyclictest566679-21memory10:15:190
501935992616,7cyclictest543964-21df_inode09:10:160
501935992616,7cyclictest521581-21/usr/sbin/munin08:05:140
501946992518,4cyclictest0-21swapper/310:55:253
501946992518,3cyclictest0-21swapper/307:45:013
501946992517,4cyclictest0-21swapper/308:30:163
501941992515,7cyclictest0-21swapper/212:20:212
50193899256,3cyclictest131rcu_preempt12:30:161
50193899254,4cyclictest131rcu_preempt07:40:121
501938992518,5cyclictest610827-21kworker/u8:3+events_unbound12:25:161
501938992517,4cyclictest0-21swapper/110:40:261
50193899250,13cyclictest0-21swapper/108:00:191
50193599259,13cyclictest591469-21mailstats11:25:220
50193599252,19cyclictest550446-21latency_hist09:30:010
501935992515,7cyclictest597576-21/usr/sbin/munin11:45:130
501935992515,7cyclictest513065-21diskstats07:40:150
501935992514,8cyclictest536272-21sshd08:46:250
501935992513,8cyclictest215-21dbus-daemon12:30:180
501946992417,4cyclictest527594-21grep08:20:283
501946992416,5cyclictest0-21swapper/307:35:013
501946992416,4cyclictest512682-21kworker/u8:2+events_unbound07:54:393
501946992415,6cyclictest519512-21sh08:00:003
501946992415,6cyclictest0-21swapper/310:45:023
501941992418,3cyclictest0-21swapper/210:05:222
501941992417,4cyclictest5521322sleep209:35:012
501941992416,5cyclictest544922-21unixbench_singl09:10:272
501941992415,5cyclictest558730-21grep09:50:242
501941992412,3cyclictest131rcu_preempt08:05:132
50193899241,7cyclictest593940-21latency_hist11:35:011
501938992417,5cyclictest615716-21kworker/u8:2+events_unbound12:39:361
501938992416,5cyclictest0-21swapper/112:00:151
501938992416,5cyclictest0-21swapper/112:00:151
501946992317,3cyclictest0-21swapper/309:55:263
501946992316,5cyclictest600464-21kworker/u8:1+events_unbound12:00:153
501946992316,5cyclictest600464-21kworker/u8:1+events_unbound12:00:153
501946992316,3cyclictest0-21swapper/312:30:183
501946992316,3cyclictest0-21swapper/311:00:163
501946992315,4cyclictest0-21swapper/309:25:163
50194199234,5cyclictest131rcu_preempt07:25:122
501941992318,3cyclictest0-21swapper/208:25:192
501941992317,4cyclictest507357-21kworker/u8:1+events_unbound07:49:382
501941992317,4cyclictest507357-21kworker/u8:1+events_unbound07:49:382
501941992317,3cyclictest0-21swapper/212:10:242
501941992317,3cyclictest0-21swapper/211:55:212
501941992317,3cyclictest0-21swapper/209:55:232
501941992317,3cyclictest0-21swapper/209:05:252
501941992317,3cyclictest0-21swapper/208:15:232
501941992315,5cyclictest440-21cron10:45:002
501941992314,6cyclictest567250-21grep10:15:232
501941992314,5cyclictest0-21swapper/210:35:262
501941992310,5cyclictest0-21swapper/207:40:212
50193899237,5cyclictest131rcu_preempt07:45:261
50193899237,5cyclictest131rcu_preempt07:45:261
501946992216,3cyclictest0-21swapper/311:35:213
501946992216,3cyclictest0-21swapper/309:10:173
501946992216,3cyclictest0-21swapper/307:20:263
501946992215,4cyclictest0-21swapper/311:45:293
501946992213,6cyclictest571257-21sed10:30:013
501941992215,5cyclictest5680732sleep210:20:162
501941992215,4cyclictest0-21swapper/209:05:012
501941992215,4cyclictest0-21swapper/207:50:152
50194199221,12cyclictest0-21swapper/211:45:132
501946992116,3cyclictest0-21swapper/310:10:253
501946992115,3cyclictest0-21swapper/312:15:263
501946992115,3cyclictest0-21swapper/308:45:233
501946992114,5cyclictest391ktimers/310:30:153
501946992114,5cyclictest391ktimers/308:05:123
501946992114,3cyclictest0-21swapper/312:30:003
501946992113,4cyclictest0-21swapper/308:00:203
501946992014,3cyclictest514866-21fschecks_time07:45:163
501946992014,3cyclictest514866-21fschecks_time07:45:163
501946992013,5cyclictest391ktimers/310:05:143
50194199209,8cyclictest592257-21date11:30:012
501941992014,4cyclictest557476-21sh09:50:002
501941992013,4cyclictest440-21cron11:15:012
501941992013,4cyclictest0-21swapper/211:00:272
501941992010,7cyclictest570590-21sed10:25:262
501946991914,2cyclictest0-21swapper/309:15:183
501946991910,6cyclictest556586-21ntpq09:45:223
501941991914,3cyclictest596066-21fschecks_time11:40:172
50194199190,17cyclictest311ktimers/207:10:182
501946991812,3cyclictest0-21swapper/308:40:133
50194199188,7cyclictest534313-21cat08:40:242
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional