You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-27 - 20:56

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot5s.osadl.org (updated Thu Nov 27, 2025 12:44:46)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1193601995646,5cyclictest1211384-21sh08:00:010
1193612995347,4cyclictest1195111-21kworker/u8:1+events_unbound07:15:323
1193612995135,6cyclictest1245648-21kworker/u8:1+events_unbound09:41:033
1193601995141,7cyclictest121ktimers/007:50:160
1193606995035,11cyclictest231ktimers/112:15:141
1193601995023,4cyclictest0-21swapper/008:05:240
1193601994825,4cyclictest1200950-21kthreadd07:30:000
1193612994639,5cyclictest1264731-21kworker/u8:2+events_unbound10:41:093
1193601994636,7cyclictest1282394-21cron11:25:000
1193606994534,8cyclictest1258031-21apt-key10:15:011
1193612994433,8cyclictest391ktimers/307:10:133
1193608994311,12cyclictest1239093-21latency_hist09:20:022
1193608994233,5cyclictest1268341-21munin-plugin-st10:45:012
1193606994212,8cyclictest131rcu_preempt07:25:121
1193612994135,4cyclictest1262977-21kworker/u8:1+events_unbound10:31:083
1193601994130,7cyclictest0-21swapper/007:20:000
1193601994129,8cyclictest1202229-21grep07:30:260
1193612994024,4cyclictest131rcu_preempt11:25:193
1193608994031,4cyclictest0-21swapper/212:20:152
1193608994014,11cyclictest1241223-21latency09:25:182
1193608994013,12cyclictest1285335-21awk11:30:252
1193608994013,12cyclictest1285335-21awk11:30:252
1193606994034,4cyclictest1283898-21kworker/u8:2+events_unbound11:31:221
1193606994034,4cyclictest1283898-21kworker/u8:2+events_unbound11:31:221
1193606994031,6cyclictest1272310-21fschecks_count10:55:151
1193606994029,7cyclictest1214383-21unixbench_singl08:05:241
119360699401,14cyclictest0-21swapper/109:45:121
1193601994029,7cyclictest1302401-21grep12:20:230
1193612993928,8cyclictest391ktimers/309:30:153
1193612993919,10cyclictest131rcu_preempt08:10:573
1193608993913,9cyclictest1206961-21meminfo07:45:192
1193608993913,11cyclictest1212150-21ls08:00:172
119360699390,14cyclictest0-21swapper/110:00:121
1193612993821,9cyclictest131rcu_preempt07:40:193
1193606993829,6cyclictest1282392-21sh11:24:591
1193606993825,9cyclictest1271514-21grep10:50:241
119360699381,14cyclictest0-21swapper/107:40:151
119360699380,14cyclictest0-21swapper/112:10:121
1193601993828,5cyclictest0-21swapper/009:20:010
1193612993729,6cyclictest1237610-21kworker/u8:1+events_unbound09:22:423
1193612993717,11cyclictest1290534-21kworker/u8:1-events_freezable_power_11:49:343
1193612993715,12cyclictest131rcu_preempt08:29:233
1193608993729,4cyclictest0-21swapper/208:15:012
1193608993728,4cyclictest1273657-21sendmail11:00:012
1193608993728,4cyclictest0-21swapper/207:15:022
1193608993725,8cyclictest1242894-21/usr/sbin/munin09:30:202
1193608993719,13cyclictest1260484-21/usr/sbin/munin10:20:192
1193608993719,13cyclictest1260484-21/usr/sbin/munin10:20:192
1193608993713,9cyclictest1277288-21/usr/sbin/munin11:10:192
1193606993714,7cyclictest131rcu_preempt11:15:121
1193612993630,4cyclictest1203989-21kworker/u8:0+events_unbound08:05:443
1193612993622,7cyclictest1285739-21kworker/u8:3+events_unbound11:38:213
1193612993617,4cyclictest131rcu_preempt10:12:193
1193608993630,3cyclictest1239152-21apt-get09:20:132
1193608993626,6cyclictest0-21swapper/208:25:012
1193608993625,7cyclictest1266626-21sendmail10:40:012
1193608993624,9cyclictest1306503-21apt-get12:35:122
1193608993624,8cyclictest1259805-21sendmail10:19:592
1193606993630,4cyclictest1304578-21munin-plugin-st12:30:001
119360199368,5cyclictest1225423-21grep08:40:160
1193601993624,9cyclictest121ktimers/012:10:130
1193601993624,6cyclictest0-21swapper/007:20:260
1193601993624,6cyclictest0-21swapper/007:20:250
1193612993525,7cyclictest0-21swapper/310:35:243
1193612993524,4cyclictest131rcu_preempt11:07:363
1193612993523,5cyclictest1254124-21kworker/u8:0+events_unbound10:09:413
1193612993520,5cyclictest131rcu_preempt11:30:123
1193612993520,5cyclictest131rcu_preempt11:30:113
1193612993510,13cyclictest40-21ksoftirqd/307:45:173
1193608993528,3cyclictest1304802-21apt-get12:30:122
1193608993527,4cyclictest0-21swapper/208:30:252
1193608993527,4cyclictest0-21swapper/207:40:372
1193608993524,7cyclictest1246413-21/usr/sbin/munin09:40:192
1193608993520,10cyclictest1275898-21/usr/sbin/munin11:05:172
1193608993520,10cyclictest1249526-21/usr/sbin/munin09:50:152
1193608993519,11cyclictest1256518-21/usr/sbin/munin10:10:172
1193606993528,5cyclictest1202282-21kworker/u8:3+events_unbound07:45:381
1193606993527,4cyclictest0-21swapper/107:55:251
1193606993519,6cyclictest1259414-21kworker/u8:0+events_unbound10:37:071
1193606993519,6cyclictest1231883-21kworker/u8:2+events_unbound09:07:001
1193601993525,6cyclictest1254287-21latency_hist10:05:010
1193601993520,11cyclictest121ktimers/011:00:130
119361299347,7cyclictest40-21ksoftirqd/309:40:013
1193612993428,4cyclictest1233646-21kworker/u8:0+events_unbound09:04:433
1193612993426,4cyclictest0-21swapper/311:11:173
1193612993423,4cyclictest131rcu_preempt09:50:263
1193612993421,5cyclictest131rcu_preempt10:15:193
1193612993418,4cyclictest131rcu_preempt11:52:383
1193608993427,3cyclictest1225286-21apt-get08:40:122
1193608993426,4cyclictest0-21swapper/208:25:142
1193608993425,4cyclictest0-21swapper/207:55:392
1193608993424,7cyclictest1232658-21/usr/sbin/munin09:00:172
1193608993424,6cyclictest0-21swapper/211:15:252
1193608993424,6cyclictest0-21swapper/210:30:002
1193608993422,7cyclictest1199583-21df07:25:122
1193608993421,10cyclictest1288199-21memory11:40:202
1193608993418,10cyclictest0-21swapper/210:00:122
1193608993417,13cyclictest1298977-21tail12:10:262
1193608993416,14cyclictest1296970-21/usr/sbin/munin12:05:212
1193608993411,6cyclictest1198345-21ntp_states07:20:212
1193608993411,6cyclictest1198345-21ntp_states07:20:212
1193606993418,5cyclictest1292253-21kworker/u8:0+events_unbound12:03:241
1193601993425,4cyclictest0-21swapper/012:35:000
1193601993423,6cyclictest0-21swapper/011:15:260
1193601993411,4cyclictest12200102cat08:25:010
119361299336,15cyclictest40-21ksoftirqd/307:55:213
1193612993324,5cyclictest0-21swapper/308:00:233
1193612993323,7cyclictest391ktimers/310:00:183
1193612993322,4cyclictest131rcu_preempt07:23:533
1193612993322,4cyclictest131rcu_preempt07:23:533
1193612993321,4cyclictest131rcu_preempt12:34:563
1193612993319,4cyclictest131rcu_preempt12:19:023
1193612993316,10cyclictest131rcu_preempt12:00:093
1193612993315,5cyclictest131rcu_preempt07:30:203
1193612993315,4cyclictest131rcu_preempt09:10:243
1193612993315,4cyclictest131rcu_preempt08:40:573
1193612993315,11cyclictest131rcu_preempt09:57:323
1193612993315,10cyclictest131rcu_preempt08:20:233
1193612993314,5cyclictest131rcu_preempt08:55:123
1193612993314,5cyclictest131rcu_preempt08:55:123
1193608993327,4cyclictest1213337-21apt-get08:05:122
1193608993326,3cyclictest1286031-21apt-get11:35:122
1193608993324,7cyclictest1281446-21nfs4_client11:20:222
1193608993323,7cyclictest1295504-21sed12:00:232
1193608993322,8cyclictest1248737-21/usr/sbin/munin09:45:252
1193608993322,8cyclictest1224743-21sshd08:36:342
1193608993322,7cyclictest1255916-21cron10:07:002
1193608993321,9cyclictest1237068-21sshd09:13:342
1193608993320,10cyclictest1201289-21/usr/sbin/munin07:30:142
1193608993319,11cyclictest1271492-21/usr/sbin/munin10:50:252
1193608993319,11cyclictest1244819-21ntp_states09:35:192
1193608993319,11cyclictest1234015-21/usr/sbin/munin09:05:092
1193608993318,9cyclictest1209273-21sed07:50:292
1193606993326,5cyclictest1262977-21kworker/u8:1+events_unbound10:30:131
1193606993325,4cyclictest1235913-21grep09:10:161
1193606993322,5cyclictest1264731-21kworker/u8:2+events_unbound10:46:141
1193601993325,4cyclictest0-21swapper/011:45:270
1193601993324,5cyclictest0-21swapper/007:10:260
1193601993323,6cyclictest0-21swapper/008:25:240
119361299326,7cyclictest40-21ksoftirqd/312:25:233
1193612993221,8cyclictest391ktimers/310:25:113
1193612993221,7cyclictest0-21swapper/308:35:233
1193612993220,5cyclictest131rcu_preempt09:05:203
1193612993219,4cyclictest131rcu_preempt11:41:143
1193612993218,7cyclictest1292253-21kworker/u8:0+events_unbound12:36:333
1193612993215,10cyclictest131rcu_preempt08:15:143
1193612993211,12cyclictest131rcu_preempt07:25:173
1193612993210,14cyclictest40-21ksoftirqd/309:30:013
1193608993222,7cyclictest1268849-21/usr/sbin/munin10:45:182
1193608993222,7cyclictest1263491-21/usr/sbin/munin10:30:132
1193608993222,7cyclictest1231813-21cron08:57:002
1193608993222,7cyclictest1231813-21cron08:56:592
1193608993222,7cyclictest1193751-21munin-node07:10:122
1193608993221,7cyclictest1300838-21sshd12:16:262
1193608993220,9cyclictest1227108-21/usr/sbin/munin08:45:182
1193608993219,10cyclictest1291994-21/usr/sbin/munin11:50:242
1193606993226,4cyclictest1216411-21kworker/u8:1+events_unbound08:15:481
1193606993224,4cyclictest0-21swapper/109:55:251
1193606993224,4cyclictest0-21swapper/109:40:161
1193606993224,4cyclictest0-21swapper/109:00:241
119360699321,5cyclictest0-21swapper/112:20:121
1193601993225,4cyclictest1262783-21grep10:25:230
1193601993225,4cyclictest0-21swapper/011:10:240
1193601993224,4cyclictest0-21swapper/009:35:230
1193601993223,5cyclictest1235063-21unixbench_singl09:05:230
1193601993222,6cyclictest0-21swapper/012:25:240
119361299318,4cyclictest391ktimers/308:45:193
1193612993120,4cyclictest131rcu_preempt08:30:193
1193612993119,5cyclictest1264731-21kworker/u8:2+events_unbound10:53:423
1193612993115,5cyclictest131rcu_preempt09:45:163
1193612993113,4cyclictest131rcu_preempt09:15:123
1193612993112,4cyclictest131rcu_preempt07:50:193
1193612993111,4cyclictest131rcu_preempt10:49:493
1193608993121,7cyclictest1303098-21/usr/sbin/munin12:25:102
1193608993121,7cyclictest1274017-21/usr/sbin/munin11:00:162
1193608993120,8cyclictest1290632-21sshd11:46:352
1193608993120,8cyclictest1283718-21sshd11:25:352
1193608993115,11cyclictest1216823-21/usr/sbin/munin08:15:142
119360699319,13cyclictest1287487-21kworker/u8:0+events_unbound11:44:521
1193606993123,4cyclictest0-21swapper/111:00:261
1193606993123,4cyclictest0-21swapper/109:35:221
1193606993122,4cyclictest0-21swapper/108:10:161
1193606993113,6cyclictest1276748-21kworker/u8:0+events_unbound11:12:021
1193601993124,4cyclictest0-21swapper/012:15:260
1193601993123,4cyclictest0-21swapper/009:25:250
1193601993118,10cyclictest121ktimers/011:35:130
119361299308,6cyclictest131rcu_preempt11:15:283
119361299308,13cyclictest40-21ksoftirqd/312:10:143
1193612993018,4cyclictest131rcu_preempt10:20:243
1193612993018,4cyclictest131rcu_preempt10:20:243
1193612993016,4cyclictest131rcu_preempt12:07:573
1193612993014,4cyclictest131rcu_preempt11:03:533
1193612993011,11cyclictest131rcu_preempt10:55:163
1193612993010,10cyclictest40-21ksoftirqd/307:35:183
1193608993020,7cyclictest1293303-21memory11:55:202
1193608993020,7cyclictest1251373-21/usr/sbin/munin09:55:172
1193608993018,7cyclictest1228687-21/usr/sbin/munin08:50:132
1193608993014,11cyclictest1204601-21latency_hist07:40:012
1193606993024,4cyclictest1240400-21kworker/u8:2+events_unbound09:26:021
1193606993023,3cyclictest0-21swapper/108:45:191
1193606993020,6cyclictest1250414-21grep09:50:261
119360699301,5cyclictest0-21swapper/108:40:121
119360199302,24cyclictest1292214-21unixbench_singl11:50:240
1193601993014,13cyclictest0-21swapper/010:50:010
1193612992916,4cyclictest131rcu_preempt11:56:383
1193612992911,4cyclictest0-21swapper/308:50:223
1193601992921,4cyclictest0-21swapper/009:00:000
1193601992921,4cyclictest0-21swapper/009:00:000
119360699286,3cyclictest131rcu_preempt07:10:121
119360699283,4cyclictest131rcu_preempt07:20:011
1193606992822,4cyclictest1204529-21apt-key07:40:001
1193606992820,5cyclictest1282137-21kworker/u8:1+events_unbound11:25:201
1193606992819,7cyclictest1264731-21kworker/u8:2+events_unbound10:40:551
1193606992818,6cyclictest1261505-21auditctl10:25:001
1193606992818,6cyclictest1261505-21auditctl10:24:591
119360699281,6cyclictest131rcu_preempt12:05:121
119360699281,5cyclictest131rcu_preempt09:20:141
119360699281,3cyclictest131rcu_preempt07:30:121
1193601992821,4cyclictest1228004-21unixbench_multi08:45:290
1193601992818,6cyclictest0-21swapper/010:55:160
1193601992817,7cyclictest1240190-21grep09:20:260
119360699274,4cyclictest131rcu_preempt07:50:161
1193606992721,4cyclictest1305994-21kworker/u8:1+events_unbound12:35:131
1193606992720,5cyclictest1221274-21kworker/u8:0+events_unbound08:30:041
1193606992720,3cyclictest1221274-21kworker/u8:0-ext4-rsv-conversion08:30:011
1193606992719,6cyclictest1293743-21timerandwakeup11:55:251
1193606992719,5cyclictest0-21swapper/108:50:181
1193606992719,5cyclictest0-21swapper/108:35:131
1193606992712,5cyclictest1250681-21kworker/u8:3+events_unbound10:08:351
1193601992722,3cyclictest1294327-21latency_hist12:00:010
1193601992718,5cyclictest0-21swapper/009:56:030
119360699269,7cyclictest131rcu_preempt08:56:421
119360699269,7cyclictest131rcu_preempt08:56:421
119360699266,6cyclictest1211714-21cpuspeed08:00:121
1193606992619,5cyclictest1195111-21kworker/u8:1+events_unbound07:20:031
1193606992619,5cyclictest1195111-21kworker/u8:1+events_unbound07:20:031
119360699261,6cyclictest1243550-21unixbench_multi09:30:271
1193606992612,4cyclictest1283898-21kworker/u8:2+events_unbound11:54:331
1193606992611,6cyclictest1233646-21kworker/u8:0+events_unbound09:19:011
1193601992619,4cyclictest1233778-21cron09:05:000
1193601992618,5cyclictest1264731-21kworker/u8:2+events_unbound10:36:080
1193601992618,4cyclictest1229694-21unixbench_multi08:50:270
1193601992616,7cyclictest0-21swapper/010:55:000
1193606992519,4cyclictest1290534-21kworker/u8:1+events_unbound11:48:081
1193606992519,4cyclictest1283898-21kworker/u8:2+events_unbound11:35:221
1193606992518,5cyclictest1257011-21kworker/u8:1+events_unbound10:15:161
119360199255,4cyclictest12493432cut09:50:010
1193601992519,4cyclictest1207509-21unixbench_multi07:45:240
1193601992518,3cyclictest0-21swapper/010:25:000
1193601992518,3cyclictest0-21swapper/010:25:000
1193601992516,6cyclictest0-21swapper/011:05:180
119360699246,4cyclictest131rcu_preempt10:25:261
1193606992417,4cyclictest1275751-21grep11:05:151
1193601992420,2cyclictest13650irq/42-30be0000.ethernet09:30:150
1193601992418,3cyclictest0-21swapper/008:00:140
1193601992418,2cyclictest0-21swapper/010:40:160
1193601992416,5cyclictest0-21swapper/012:00:230
1193601992416,4cyclictest1250420-21head09:50:260
1193601992416,4cyclictest0-21swapper/012:40:000
1193601992414,7cyclictest0-21swapper/007:40:010
119360199241,19cyclictest0-21swapper/012:07:210
119360199241,10cyclictest1264924-21basename10:34:590
119360199240,20cyclictest0-21swapper/011:42:310
119360199240,20cyclictest0-21swapper/010:12:230
119360199240,20cyclictest0-21swapper/008:32:220
119360699234,4cyclictest0-21swapper/112:30:151
119360699234,3cyclictest131rcu_preempt08:20:101
1193601992317,3cyclictest0-21swapper/008:10:250
119360199231,18cyclictest0-21swapper/010:16:220
119360199230,21cyclictest0-21swapper/008:19:510
119360199230,19cyclictest0-21swapper/011:27:310
119360199229,8cyclictest215-21dbus-daemon11:31:210
119360199229,8cyclictest215-21dbus-daemon11:31:200
1193601992216,2cyclictest0-21swapper/009:10:210
1193601992214,5cyclictest0-21swapper/008:40:010
119360199211,4cyclictest1205913-21sshd07:40:360
119360199201,3cyclictest0-21swapper/009:44:290
1193601992010,7cyclictest1255007-21cat10:05:180
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional