You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-02-25 - 08:50

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot6s.osadl.org (updated Sun Feb 25, 2024 00:43:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1302621450,8sleep112624-21/usr/sbin/munin23:52:221
32725911000,1ptp4l241ktimersoftd/104:25:161
160152990,8sleep01461099cyclictest23:57:400
128872966,26sleep012897-21/usr/sbin/munin23:52:200
221062920,9sleep11461199cyclictest02:22:401
3272591900,1ptp4l241ktimersoftd/103:54:191
1461099852,15cyclictest14915-21sshd02:12:500
3272591830,1ptp4l241ktimersoftd/103:36:381
14610998211,37cyclictest2795-21fschecks_time00:42:230
3272591800,1ptp4l241ktimersoftd/104:01:381
3272591790,1ptp4l241ktimersoftd/105:19:191
3272591790,1ptp4l241ktimersoftd/102:31:281
3272591780,1ptp4l241ktimersoftd/102:02:221
210782780,3sleep10-21swapper/105:13:051
3272591770,1ptp4l241ktimersoftd/102:38:391
3272591760,2ptp4l22601-21kworker/1:102:35:381
3272591760,21ptp4l13649-21kworker/1:203:04:471
1461099760,7cyclictest15566-21ssh03:39:280
1461099760,6cyclictest4740-21sendmail_mailqu02:42:350
3272591750,4ptp4l12312-21kworker/1:202:17:181
3272591750,3ptp4l4679-21kworker/1:203:50:391
3272591750,3ptp4l29739-21kworker/1:104:02:331
1461099750,8cyclictest9025-21df00:57:220
1461099750,10cyclictest14650-21ssh05:03:170
1461099740,7cyclictest21872-21ssh04:30:290
1461099740,7cyclictest10717-21sh03:32:400
1461099730,8cyclictest9367-21cstates04:57:290
3272591720,3ptp4l6366-21kworker/1:004:21:191
3272591720,1ptp4l241ktimersoftd/103:39:441
1461099720,7cyclictest15712-21/usr/sbin/munin01:12:260
3272591710,15ptp4l241ktimersoftd/103:12:511
1461099710,8cyclictest3198-21ssh04:47:530
1461099710,7cyclictest26666-21apt-get04:37:210
1461099710,7cyclictest22066-21ssh05:16:210
1461099701,9cyclictest31021-21ssh02:35:020
3272591690,15ptp4l241ktimersoftd/104:42:121
3272591690,14ptp4l241ktimersoftd/105:26:451
1461099695,8cyclictest5643-21apt-get03:27:210
1461099694,9cyclictest31374-21snmp_o5powerctl01:47:390
1461099683,6cyclictest23853-21snmp_o5powerctl05:17:440
1461099682,7cyclictest18043-21ssh04:25:280
1461099672,7cyclictest22903-21ssh02:23:480
1461099671,8cyclictest19110-21ssh03:43:490
1461099671,14cyclictest18459-21ssh05:10:050
1461099670,6cyclictest24226-21snmp_o5powerctl04:32:490
3272591660,15ptp4l241ktimersoftd/105:02:161
1461099662,7cyclictest27242-21snmpd00:53:110
1461099661,9cyclictest5299-21ssh04:52:040
1461099661,8cyclictest13787-21ssh04:19:230
1461099661,7cyclictest32322-21ssh03:18:540
1461099652,9cyclictest25894-21/usr/sbin/munin01:37:190
1461099652,7cyclictest21896-21ssh03:06:070
1461099651,9cyclictest2496-21ssh02:40:210
1461099651,8cyclictest25584-21ssh03:10:520
1461099651,8cyclictest17139-21ssh02:58:510
1461099651,7cyclictest5631-21ssh04:08:360
1461099651,14cyclictest10315-21ssh04:15:070
1461099642,9cyclictest28053-21ssh02:31:540
1461099642,7cyclictest28689-21ssh03:14:140
1461099642,7cyclictest23808-21ssh03:50:530
1461099641,7cyclictest32695-21ssh04:44:080
14610996412,31cyclictest12078-21grep01:02:340
1461099631,8cyclictest21938-21/usr/sbin/munin00:12:260
1461099631,8cyclictest1761-21diskmemload03:56:290
1461099631,7cyclictest10809-21ssh02:07:380
1461099630,8cyclictest30696-21ssh03:59:080
1461099622,7cyclictest2855-21ssh04:06:000
1461099622,7cyclictest13964-21ssh02:55:220
1461099621,7cyclictest3613-21ssh03:23:410
1461099612,8cyclictest4617-21apt-get00:47:170
1461099612,7cyclictest27242-21snmpd01:36:100
3272591580,1ptp4l241ktimersoftd/102:52:161
1461099585,44cyclictest1524-21snmp_o5powerctl00:37:360
3272591570,3ptp4l1448-21kworker/1:001:57:271
1461099570,8cyclictest3416-21ls01:57:420
1461099570,8cyclictest19927-21grep01:22:220
3272591560,3ptp4l21293-21kworker/1:104:48:551
3272591560,1ptp4l241ktimersoftd/104:57:581
1461099562,6cyclictest28086-21apt-get01:42:180
1461099562,6cyclictest10184-21ssh02:50:260
1461099562,49cyclictest7027-21ssh02:02:350
1461099560,7cyclictest18451-21snmp_o5powerctl01:17:340
1461099560,6cyclictest18013-21open_files02:17:310
3272591550,3ptp4l19541-21kworker/1:201:52:391
14610995514,35cyclictest31113-21irqrtprio00:32:230
1461099550,7cyclictest21331-21sed00:07:410
3272591540,6ptp4l241ktimersoftd/100:37:301
3272591530,3ptp4l4679-21kworker/1:203:28:461
1461099534,11cyclictest29536-21snmp_o5powerctl00:27:400
14610995315,32cyclictest16991-21if_enp4s000:01:570
1461099531,47cyclictest28299-21ssh05:25:150
3272591520,4ptp4l13649-21kworker/1:203:07:401
3272591520,1ptp4l241ktimersoftd/101:42:171
14611995213,16cyclictest25-21ksoftirqd/104:38:441
1461099521,6cyclictest23510-21taskset01:30:510
1461099520,6cyclictest26421-21df_inode00:22:210
3272591510,1ptp4l241ktimersoftd/101:12:131
14610995111,35cyclictest24623-21if_enp1s0.100:17:270
3272591500,1ptp4l241ktimersoftd/102:50:441
1461099505,10cyclictest32326-21apt-get01:52:170
14610995013,5cyclictest14269-21switchtime01:07:400
3272591490,1ptp4l241ktimersoftd/105:11:211
3272591490,1ptp4l241ktimersoftd/104:33:171
3272591490,1ptp4l241ktimersoftd/104:27:551
3272591490,1ptp4l241ktimersoftd/104:14:331
3272591490,1ptp4l241ktimersoftd/103:46:121
3272591490,1ptp4l241ktimersoftd/102:42:251
3272591490,1ptp4l241ktimersoftd/102:09:411
3272591490,1ptp4l241ktimersoftd/101:03:421
3272591490,1ptp4l241ktimersoftd/100:42:281
161412490,7sleep11461199cyclictest02:15:431
14611994927,10cyclictest25-21ksoftirqd/100:02:231
3272591480,1ptp4l241ktimersoftd/104:07:271
3272591480,1ptp4l241ktimersoftd/103:24:271
3272591480,1ptp4l241ktimersoftd/103:00:521
3272591480,1ptp4l241ktimersoftd/100:57:391
3272591470,1ptp4l241ktimersoftd/103:19:141
3272591470,1ptp4l241ktimersoftd/101:51:321
3272591470,1ptp4l241ktimersoftd/101:26:341
3272591470,1ptp4l241ktimersoftd/100:55:401
3272591470,1ptp4l241ktimersoftd/100:07:401
3272591460,1ptp4l241ktimersoftd/100:32:141
3272591460,1ptp4l241ktimersoftd/100:22:201
3272591450,1ptp4l241ktimersoftd/100:13:031
1461199455,5cyclictest25-21ksoftirqd/100:27:241
3272591440,1ptp4l241ktimersoftd/104:52:361
3272591430,1ptp4l241ktimersoftd/101:10:231
3272591430,14ptp4l241ktimersoftd/101:31:541
3272591410,1ptp4l241ktimersoftd/101:37:121
3272591410,1ptp4l241ktimersoftd/100:20:531
14611994010,5cyclictest25-21ksoftirqd/101:32:261
3272591380,13ptp4l241ktimersoftd/100:00:111
3272591370,1ptp4l241ktimersoftd/100:51:131
3272591330,10ptp4l241ktimersoftd/101:20:031
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional