You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-09-30 - 13:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot5s (updated Sat Sep 30, 2023 00:44:56)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2185505995848,5cyclictest2291562-21idleruntime-cro00:15:012
2185492995640,11cyclictest121ktimers/000:25:180
2185510995140,7cyclictest2284086-21grep23:50:423
2185510994834,11cyclictest391ktimers/320:20:203
2185505994842,4cyclictest2275618-21kworker/u8:3+events_unbound23:32:362
2185510994536,5cyclictest2232528-21ls21:25:003
2185510994536,5cyclictest2232528-21ls21:24:593
2185492994336,5cyclictest2239209-21kworker/u8:2+events_unbound22:12:270
2185492994230,9cyclictest2251489-21sendmail22:20:000
2185492994230,9cyclictest2251489-21sendmail22:19:590
2185510994131,6cyclictest2228644-21grep21:10:343
2185510994129,9cyclictest391ktimers/320:40:213
2185505994134,5cyclictest2213659-21kworker/u8:0+events_unbound20:57:132
2185497994133,5cyclictest2294979-21ls00:25:001
2185497994133,5cyclictest2294979-21ls00:25:001
2185497994132,7cyclictest231ktimers/122:10:211
2185497994132,6cyclictest2220582-21munin-plugin-st20:50:001
2185497994132,6cyclictest2220582-21munin-plugin-st20:50:001
218550599403,5cyclictest131rcu_preempt00:35:162
2185505994029,7cyclictest0-21swapper/222:20:012
2185505994029,7cyclictest0-21swapper/222:20:012
2185505994025,12cyclictest311ktimers/223:40:182
2185510993932,4cyclictest0-21swapper/320:20:013
2185492993931,5cyclictest2258099-21unixbench_singl22:35:350
2185510993829,6cyclictest2214160-21fschecks_count20:30:233
2185510993828,6cyclictest2274491-21fschecks_count23:25:233
218550599386,5cyclictest131rcu_preempt21:15:202
2185505993831,5cyclictest2248297-21kworker/u8:1+events_unbound22:27:292
2185505993831,5cyclictest2248297-21kworker/u8:1+events_unbound22:27:292
2185497993827,7cyclictest2271911-21grep23:15:341
2185492993832,4cyclictest2204876-21kworker/u8:4+events_unbound20:12:030
2185492993832,4cyclictest2204876-21kworker/u8:4+events_unbound20:12:020
2185510993730,5cyclictest2236353-21kworker/u8:0+events_unbound23:07:343
2185510993727,6cyclictest2279634-21fschecks_time23:40:223
2185510993726,7cyclictest2285814-21grep23:55:353
2185505993730,5cyclictest2251938-21kworker/u8:3+events_unbound22:30:192
2185492993731,4cyclictest2236357-21kworker/u8:3+events_unbound21:37:200
2185492993731,4cyclictest2236357-21kworker/u8:3+events_unbound21:37:190
2185492993730,5cyclictest2277284-21kworker/u8:4+events_unbound23:42:390
2185492993730,5cyclictest2218720-21kworker/u8:2+events_unbound21:22:180
2185492993730,5cyclictest2218720-21kworker/u8:2+events_unbound21:22:180
2185492993730,4cyclictest2185490-21cyclictest22:25:330
2185492993730,4cyclictest2185490-21cyclictest22:25:330
2185492993727,7cyclictest2235930-21grep21:35:000
2185510993630,4cyclictest2201550-21kworker/u8:1+events_unbound19:56:583
2185510993625,6cyclictest0-21swapper/321:50:333
2185505993627,4cyclictest0-21swapper/222:35:362
2185492993629,4cyclictest2291560-21grep00:15:010
2185492993626,6cyclictest2242563-21grep21:50:330
2185510993527,5cyclictest2272354-21cron23:20:003
2185505993529,4cyclictest2277284-21kworker/u8:4+events_unbound23:37:372
2185505993524,7cyclictest2237365-21grep21:35:342
2185505993524,7cyclictest2237365-21grep21:35:342
2185497993527,5cyclictest2203768-21apt-get20:00:191
2185497993525,6cyclictest0-21swapper/120:30:351
2185497993522,10cyclictest2229602-21/usr/sbin/munin21:15:251
2185492993520,6cyclictest0-21swapper/020:40:010
2185510993424,6cyclictest0-21swapper/300:10:003
2185505993427,5cyclictest2204876-21kworker/u8:4+events_unbound20:27:072
2185505993426,4cyclictest0-21swapper/222:45:342
2185505993426,4cyclictest0-21swapper/200:20:242
2185505993426,4cyclictest0-21swapper/200:20:232
2185505993425,6cyclictest2189868-21fschecks_count19:20:262
2185497993427,4cyclictest0-21swapper/119:15:011
2185497993426,5cyclictest0-21swapper/122:20:001
2185497993426,5cyclictest0-21swapper/122:19:591
2185497993426,4cyclictest0-21swapper/120:05:201
2185497993420,10cyclictest2245490-21ntp_states22:00:321
218549299344,4cyclictest131rcu_preempt00:30:260
2185492993423,7cyclictest2228710-21grep21:10:340
2185510993324,6cyclictest2300306-21sendmail00:39:593
2185510993322,6cyclictest0-21swapper/322:45:353
2185510993322,6cyclictest0-21swapper/319:45:233
2185505993326,4cyclictest2242929-21kworker/u8:3+events_unbound21:55:242
2185505993323,6cyclictest2266587-21grep23:00:332
2185497993326,3cyclictest2290093-21apt-get00:10:181
2185497993325,4cyclictest0-21swapper/120:15:361
2185497993324,5cyclictest0-21swapper/119:15:271
2185497993323,6cyclictest0-21swapper/121:55:331
2185497993320,10cyclictest2258767-21munin-node22:40:171
2185497993320,10cyclictest2258767-21munin-node22:40:171
2185497993320,10cyclictest2236472-21/usr/sbin/munin21:35:201
2185497993320,10cyclictest2236472-21/usr/sbin/munin21:35:201
2185497993320,10cyclictest2233213-21/usr/sbin/munin21:25:271
2185497993320,10cyclictest2231593-21/usr/sbin/munin21:20:271
2185497993320,10cyclictest2231593-21/usr/sbin/munin21:20:271
2185497993320,10cyclictest2215816-21/usr/sbin/munin20:35:241
2185497993320,10cyclictest2200656-21/usr/sbin/munin19:50:291
2185497993319,11cyclictest2193447-21/usr/sbin/munin19:30:261
218549299336,6cyclictest131rcu_preempt19:55:190
218549299334,5cyclictest131rcu_preempt23:45:180
218549299333,26cyclictest2209862-21grep20:15:360
2185492993324,7cyclictest2266721-21kworker/u8:2+events_unbound23:32:390
2185492993323,6cyclictest0-21swapper/000:25:000
2185492993323,6cyclictest0-21swapper/000:24:590
2185510993224,6cyclictest2251938-21kworker/u8:3+events_unbound22:24:173
2185510993223,7cyclictest2270417-21kworker/u8:1+events_unbound23:23:233
218550599324,6cyclictest131rcu_preempt20:35:202
218549799327,22cyclictest2264677-21grep22:55:301
2185497993224,4cyclictest2284761-21apt-get23:55:201
2185497993222,7cyclictest2253771-21/usr/sbin/munin22:25:231
2185497993222,7cyclictest2253771-21/usr/sbin/munin22:25:231
2185497993222,7cyclictest2222515-21/usr/sbin/munin20:55:201
2185497993219,10cyclictest2295816-21/usr/sbin/munin00:25:281
2185497993219,10cyclictest2280563-21cron23:41:001
2185497993219,10cyclictest2257901-21/usr/sbin/munin22:35:341
2185497993219,10cyclictest2207512-21/usr/sbin/munin20:10:281
2185497993219,10cyclictest2207512-21/usr/sbin/munin20:10:281
2185497993219,10cyclictest2199043-21/usr/sbin/munin19:45:311
2185497993211,17cyclictest2299269-21chrt00:35:241
218549299323,4cyclictest131rcu_preempt21:45:180
218549299323,4cyclictest131rcu_preempt21:45:180
2185492993226,4cyclictest2166565-21kworker/u8:3+events_unbound19:21:510
218549299321,5cyclictest131rcu_preempt23:15:190
218549299321,4cyclictest131rcu_preempt00:35:160
2185510993123,5cyclictest2251938-21kworker/u8:3+events_unbound22:35:353
2185510993123,4cyclictest0-21swapper/322:27:283
2185510993123,4cyclictest0-21swapper/322:27:283
2185510993121,6cyclictest0-21swapper/322:00:343
2185510993120,7cyclictest0-21swapper/320:35:333
2185510993120,6cyclictest0-21swapper/323:00:013
2185505993124,5cyclictest2251938-21kworker/u8:3+events_unbound22:22:292
2185505993123,5cyclictest2268587-21kworker/u8:3+events_unbound23:12:132
2185505993123,5cyclictest2268587-21kworker/u8:3+events_unbound23:12:122
2185505993123,4cyclictest2296672-21apt00:29:592
2185505993123,4cyclictest0-21swapper/220:40:372
2185505993123,4cyclictest0-21swapper/219:45:352
2185505993121,6cyclictest2215031-21grep20:30:352
2185505993120,7cyclictest0-21swapper/200:00:002
2185505993119,9cyclictest2203317-21sendmail20:00:002
2185497993122,6cyclictest2239915-21/usr/sbin/munin21:45:221
2185497993122,6cyclictest2239915-21/usr/sbin/munin21:45:211
2185497993121,7cyclictest2197278-21/usr/sbin/munin19:40:271
2185497993120,8cyclictest2267502-21/usr/sbin/munin23:05:201
2185497993118,9cyclictest0-21swapper/123:25:241
2185497993118,10cyclictest2272898-21/usr/sbin/munin23:20:241
2185497993117,11cyclictest2185490-21cyclictest20:20:191
2185497993115,13cyclictest2276806-21/usr/sbin/munin23:30:341
2185497993115,11cyclictest2202898-21grep19:55:331
2185492993124,4cyclictest2205232-21idleruntime-cro20:05:010
2185492993123,5cyclictest2261773-21kworker/u8:2+events_unbound22:53:140
2185510993023,5cyclictest2189229-21kworker/u8:2+events_unbound19:34:043
2185510993022,4cyclictest0-21swapper/323:45:343
2185510993022,4cyclictest0-21swapper/323:00:323
2185510993021,7cyclictest2242929-21kworker/u8:3+events_unbound22:06:073
2185510993021,6cyclictest0-21swapper/321:15:233
2185510993019,6cyclictest0-21swapper/300:00:423
2185505993022,4cyclictest0-21swapper/223:20:002
2185505993022,4cyclictest0-21swapper/221:30:332
2185505993022,4cyclictest0-21swapper/219:25:352
2185505993022,4cyclictest0-21swapper/219:25:352
2185505993021,5cyclictest0-21swapper/219:15:012
2185497993025,3cyclictest2217418-21apt-get20:40:221
2185497993024,3cyclictest2291910-21df_inode00:15:251
2185497993024,3cyclictest2283061-21apt-get23:50:201
2185497993023,4cyclictest2297125-21apt-get00:30:251
2185497993021,7cyclictest2260471-21apt-get22:45:211
2185497993020,7cyclictest2288500-21/usr/sbin/munin00:05:221
2185497993020,7cyclictest2266704-21cron23:01:011
2185497993020,7cyclictest2247068-21/usr/sbin/munin22:05:271
2185497993020,7cyclictest2242016-21/usr/sbin/munin21:50:261
2185497993020,7cyclictest2190227-21/usr/sbin/munin19:20:301
2185497993017,10cyclictest2270571-21cron23:14:011
2185497993017,10cyclictest2270571-21cron23:14:001
2185497993017,10cyclictest2226193-21/usr/sbin/munin21:05:211
2185497993017,10cyclictest2195122-21df19:35:201
2185497993017,10cyclictest2195122-21df19:35:191
2185497993012,15cyclictest2225251-21grep21:00:351
218549299303,4cyclictest131rcu_preempt19:30:200
2185492993023,5cyclictest2213659-21kworker/u8:0+events_unbound21:03:200
2185492993023,5cyclictest2213659-21kworker/u8:0+events_unbound20:29:020
2185492993023,4cyclictest0-21swapper/000:20:000
2185492993023,4cyclictest0-21swapper/000:10:000
2185492993023,3cyclictest0-21swapper/022:40:320
2185492993023,3cyclictest0-21swapper/022:40:310
2185492993021,7cyclictest2236353-21kworker/u8:0+events_unbound23:10:460
2185492993021,7cyclictest2236353-21kworker/u8:0+events_unbound23:10:460
2185492993020,6cyclictest2196477-21auditctl19:40:010
2185492993020,6cyclictest2196477-21auditctl19:40:010
218549299301,4cyclictest131rcu_preempt23:55:190
218549299301,14cyclictest0-21swapper/019:40:270
2185510992924,2cyclictest2233957-21kworker/u8:1+events_unbound21:42:233
2185510992921,4cyclictest0-21swapper/320:05:323
2185510992918,6cyclictest0-21swapper/320:25:343
2185510992915,8cyclictest0-21swapper/319:30:013
2185510992915,8cyclictest0-21swapper/319:30:003
2185505992923,4cyclictest2213659-21kworker/u8:0+events_unbound21:07:262
2185505992922,5cyclictest2272619-21kworker/u8:0+events_unbound00:05:222
2185505992922,5cyclictest2251938-21kworker/u8:3+events_unbound22:42:182
2185505992922,5cyclictest2251938-21kworker/u8:3+events_unbound22:42:172
2185505992922,5cyclictest2239209-21kworker/u8:2+events_unbound22:11:142
2185505992922,3cyclictest0-21swapper/219:40:332
2185505992921,6cyclictest2236353-21kworker/u8:0+events_unbound23:08:122
2185505992921,4cyclictest0-21swapper/223:50:002
2185497992921,5cyclictest2227642-21apt-get21:10:181
2185497992921,4cyclictest0-21swapper/121:35:001
2185497992920,6cyclictest2287851-21sshd00:01:321
2185497992919,7cyclictest2255620-21/usr/sbin/munin22:30:211
2185497992918,8cyclictest2212537-21/usr/sbin/munin20:25:231
2185497992917,8cyclictest2191403-21/usr/sbin/munin19:25:241
2185497992917,8cyclictest2191403-21/usr/sbin/munin19:25:241
2185497992917,10cyclictest2253032-21uptime22:20:401
2185497992912,13cyclictest2281930-21grep23:45:311
2185492992924,3cyclictest13650irq/42-30be0000.ethernet21:25:300
2185492992923,4cyclictest2266721-21kworker/u8:2+events_unbound23:04:340
2185492992923,4cyclictest2251938-21kworker/u8:3+events_unbound22:47:320
2185492992922,5cyclictest2242929-21kworker/u8:3+events_unbound21:55:420
2185492992921,6cyclictest2166565-21kworker/u8:3+events_unbound19:28:460
2185492992921,6cyclictest2166565-21kworker/u8:3+events_unbound19:28:460
2185492992921,4cyclictest2230832-21sendmail-msp21:19:590
2185492992921,4cyclictest0-21swapper/023:50:420
2185492992921,4cyclictest0-21swapper/021:05:320
2185492992919,6cyclictest2273624-21grep23:20:330
2185510992821,5cyclictest2277284-21kworker/u8:4+events_unbound23:39:083
2185510992820,4cyclictest0-21swapper/319:50:333
2185510992818,6cyclictest0-21swapper/322:15:363
2185510992818,6cyclictest0-21swapper/322:15:353
2185510992817,7cyclictest0-21swapper/322:50:333
2185510992816,8cyclictest0-21swapper/319:15:013
2185505992822,4cyclictest2272619-21kworker/u8:0+events_unbound00:18:122
2185505992822,4cyclictest2189229-21kworker/u8:2+events_unbound20:16:112
2185505992821,5cyclictest2239209-21kworker/u8:2+events_unbound21:48:052
2185505992821,5cyclictest2239209-21kworker/u8:2+events_unbound21:48:042
2185505992821,5cyclictest2236353-21kworker/u8:0+events_unbound21:40:052
2185505992821,5cyclictest2166565-21kworker/u8:3+events_unbound19:19:122
2185505992820,4cyclictest0-21swapper/221:10:232
2185505992817,7cyclictest2245961-21grep22:00:352
2185497992818,7cyclictest2238843-21/usr/sbin/munin21:40:311
2185497992817,8cyclictest2222013-21cron20:52:001
218549299287,5cyclictest131rcu_preempt22:00:210
2185492992822,4cyclictest2272619-21kworker/u8:0+events_unbound23:35:020
2185492992822,4cyclictest2211626-21kworker/u8:1+events_unbound20:42:340
2185492992821,5cyclictest2204876-21kworker/u8:4+events_unbound20:55:280
2185492992821,5cyclictest2180147-21kworker/u8:1+events_unbound19:12:170
2185492992821,5cyclictest2159738-21kworker/u8:0+events_unbound19:17:260
2185492992820,6cyclictest2266721-21kworker/u8:2+events_unbound23:25:290
2185510992720,5cyclictest2251938-21kworker/u8:3+events_unbound22:42:313
2185510992720,5cyclictest2251938-21kworker/u8:3+events_unbound22:42:313
2185510992720,5cyclictest2180147-21kworker/u8:1+events_unbound19:18:593
2185510992717,6cyclictest0-21swapper/319:40:013
2185510992717,6cyclictest0-21swapper/319:40:013
2185505992720,5cyclictest2270417-21kworker/u8:1+events_unbound23:25:082
2185505992720,5cyclictest2233957-21kworker/u8:1+events_unbound21:28:592
2185505992720,5cyclictest2189229-21kworker/u8:2+events_unbound20:05:092
2185505992720,5cyclictest2166565-21kworker/u8:3+events_unbound20:02:492
2185505992718,6cyclictest311ktimers/219:35:192
2185505992718,6cyclictest311ktimers/219:35:182
2185497992720,4cyclictest2277729-21apt-get23:35:201
2185492992721,4cyclictest2236353-21kworker/u8:0+events_unbound22:23:070
2185492992721,4cyclictest2194286-21kworker/u8:0+events_unbound19:53:390
2185492992718,5cyclictest2239085-21sed21:40:320
2185492992718,5cyclictest2199501-21grep19:45:340
2185510992620,4cyclictest2218720-21kworker/u8:2+events_unbound20:54:173
2185510992619,5cyclictest2290645-21ntp_offset00:10:293
2185510992616,6cyclictest0-21swapper/320:00:373
2185510992616,6cyclictest0-21swapper/319:40:333
2185510992615,6cyclictest0-21swapper/300:30:013
2185510992615,6cyclictest0-21swapper/300:20:353
2185510992615,6cyclictest0-21swapper/300:20:343
2185505992620,4cyclictest2272619-21kworker/u8:0+events_unbound23:20:342
2185505992620,4cyclictest2236353-21kworker/u8:0+events_unbound22:52:112
2185505992620,4cyclictest2194286-21kworker/u8:0+events_unbound19:51:572
2185505992619,5cyclictest2204876-21kworker/u8:4+events_unbound21:20:312
2185505992619,5cyclictest2204876-21kworker/u8:4+events_unbound21:20:302
2185505992619,4cyclictest0-21swapper/220:20:342
2185505992615,7cyclictest2286890-21grep00:00:312
218549299264,3cyclictest131rcu_preempt23:00:010
2185492992619,3cyclictest0-21swapper/023:10:010
2185492992615,8cyclictest2287243-21ntpq00:00:370
2185492992614,7cyclictest213-21dbus-daemon22:07:250
2185510992518,5cyclictest2239209-21kworker/u8:2+events_unbound21:48:183
2185510992518,5cyclictest2239209-21kworker/u8:2+events_unbound21:48:183
2185510992515,6cyclictest0-21swapper/323:15:013
2185510992515,6cyclictest0-21swapper/323:15:013
2185510992515,6cyclictest0-21swapper/322:10:353
2185510992514,7cyclictest0-21swapper/321:55:223
2185505992519,4cyclictest2297127-21kworker/u8:0+events_unbound00:32:232
2185505992519,4cyclictest2189229-21kworker/u8:2+events_unbound19:33:562
2185505992517,4cyclictest0-21swapper/222:05:202
218550599250,6cyclictest0-21swapper/223:50:202
2185492992519,3cyclictest0-21swapper/020:30:340
2185492992518,3cyclictest0-21swapper/022:30:310
2185492992517,5cyclictest0-21swapper/020:50:010
2185492992517,5cyclictest0-21swapper/020:50:010
2185510992418,3cyclictest0-21swapper/319:20:363
2185510992416,4cyclictest2277488-21latency_hist23:35:003
2185510992414,6cyclictest0-21swapper/320:55:243
2185505992418,4cyclictest2213659-21kworker/u8:0+events_unbound20:48:152
2185505992418,4cyclictest2213659-21kworker/u8:0+events_unbound20:48:152
2185505992417,5cyclictest2248297-21kworker/u8:1+events_unbound22:59:552
2185510992315,4cyclictest0-21swapper/300:35:013
2185505992316,5cyclictest2218720-21kworker/u8:2+events_unbound21:01:122
2185505992312,7cyclictest1-21systemd20:12:032
2185505992312,7cyclictest1-21systemd20:12:032
2185492992317,4cyclictest2204876-21kworker/u8:4+events_unbound20:21:180
2185492992316,3cyclictest0-21swapper/020:05:060
2185492992312,9cyclictest2221733-21wc20:50:330
2185510992212,7cyclictest2236905-21ntpq21:35:283
2185510992212,7cyclictest2236905-21ntpq21:35:283
2185505992215,5cyclictest2218720-21kworker/u8:2+events_unbound20:50:112
2185510992115,3cyclictest0-21swapper/322:30:323
2185510992115,3cyclictest0-21swapper/321:00:223
2185510992115,3cyclictest0-21swapper/300:15:273
2185510992114,5cyclictest2218720-21kworker/u8:2+events_unbound21:29:103
2185510992110,8cyclictest2219695-21ntp_states20:45:343
2185510992110,8cyclictest2219695-21ntp_states20:45:343
2185510992011,6cyclictest0-21swapper/321:05:213
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional