You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-12-02 - 22:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #4, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack4slot7s.osadl.org (updated Sat Dec 02, 2023 12:43:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
238642494427,44sleep00-21swapper/007:06:440
237852490430,38sleep10-21swapper/107:05:521
255342770,1sleep025533-21open_files11:30:100
90152550,2sleep10-21swapper/109:15:111
24309993721,9cyclictest0-21swapper/110:25:021
2430999274,14cyclictest5770-21diskmemload11:10:031
2430899272,18cyclictest5770-21diskmemload09:39:540
138672260,2sleep10-21swapper/108:05:031
2430999253,14cyclictest5770-21diskmemload10:04:591
2430899252,18cyclictest22829-21snmpd07:24:560
2430999248,4cyclictest141rcu_preempt09:00:001
2430999244,4cyclictest141rcu_preempt09:44:581
2430999243,11cyclictest5770-21diskmemload09:30:021
2430999237,9cyclictest22-21ksoftirqd/110:19:391
24309992318,3cyclictest12357-21turbostat.cron12:29:381
23802230,1sleep00-21swapper/009:00:020
2430999226,14cyclictest7676-21awk09:14:381
2430999225,5cyclictest141rcu_preempt07:10:031
2430999224,9cyclictest0-21swapper/107:55:001
3282210,2chrt0-21swapper/108:55:031
2430999219,5cyclictest0-21swapper/110:39:541
2430999218,6cyclictest141rcu_preempt11:00:171
2430999217,6cyclictest141rcu_preempt08:42:041
2430999214,7cyclictest0-21swapper/107:45:031
2430999214,6cyclictest0-21swapper/111:14:591
2430999213,5cyclictest0-21swapper/110:50:021
2430999213,14cyclictest25035-21df_inode11:30:031
24309992111,8cyclictest11666-21/usr/sbin/munin09:25:081
24309992110,3cyclictest22-21ksoftirqd/108:19:381
136602210,1sleep00-21swapper/012:30:150
2430999209,3cyclictest22-21ksoftirqd/112:30:131
2430999209,3cyclictest22-21ksoftirqd/108:10:091
2430999208,6cyclictest141rcu_preempt09:09:001
2430999208,4cyclictest141rcu_preempt11:25:191
2430999207,9cyclictest21585-21fschecks_count09:50:091
2430999205,2cyclictest141rcu_preempt12:35:231
2430999204,9cyclictest0-21swapper/109:59:541
2430999204,4cyclictest141rcu_preempt11:59:521
2430999204,10cyclictest0-21swapper/107:49:571
2430999202,2cyclictest2250ksoftirqd/109:35:031
24309992011,7cyclictest22829-21snmpd12:15:291
24309992011,7cyclictest22829-21snmpd11:38:121
24309992011,7cyclictest22829-21snmpd07:22:591
24309992011,7cyclictest22829-21snmpd07:22:591
2430999201,13cyclictest22-21ksoftirqd/111:40:011
2430999200,4cyclictest0-21swapper/109:20:481
2430899203,2cyclictest66241ntp_states07:45:150
24308992011,7cyclictest22829-21snmpd09:50:180
2430899201,10cyclictest17099-21grep11:10:050
24308992010,8cyclictest22829-21snmpd08:15:100
2430899200,8cyclictest15692if_enp2s010:25:090
2430999199,8cyclictest22829-21snmpd11:21:031
2430999199,3cyclictest22-21ksoftirqd/110:30:081
2430999198,9cyclictest6581-1kworker/u5:212:10:111
2430999198,8cyclictest22829-21snmpd07:29:381
2430999195,8cyclictest0-21swapper/112:04:571
2430999192,11cyclictest22-21ksoftirqd/108:29:571
2430999191,5cyclictest8165-21ntp_states10:45:101
24309991913,4cyclictest31250-21missed_timers08:50:071
2430999191,12cyclictest30897-21hddtemp_smartct11:45:061
24309991910,7cyclictest22829-21snmpd07:33:541
2430899199,8cyclictest14233-21ntp_states08:05:090
2430899191,6cyclictest0-21swapper/011:44:380
2430899191,4cyclictest0-21swapper/011:50:020
24308991911,6cyclictest470-21dbus-daemon07:39:380
24308991911,6cyclictest22829-21snmpd12:27:440
24308991911,6cyclictest22829-21snmpd10:23:580
24308991911,6cyclictest22829-21snmpd10:11:190
24308991911,6cyclictest22829-21snmpd10:08:460
24308991911,6cyclictest22829-21snmpd08:00:540
24308991911,4cyclictest0-21swapper/011:20:030
24308991910,7cyclictest7342-21grep12:10:180
24308991910,7cyclictest22829-21snmpd10:55:110
24308991910,7cyclictest22829-21snmpd10:49:560
24308991910,7cyclictest22829-21snmpd08:41:290
24308991910,7cyclictest20080-21ntpq09:45:110
2430999189,7cyclictest2517-21cat07:35:061
2430999189,7cyclictest2115-21seq11:58:471
2430999184,8cyclictest22-21ksoftirqd/107:39:581
2430999184,8cyclictest0-21swapper/107:14:591
2430999182,14cyclictest22283-21grep08:25:161
2430999182,12cyclictest24851-21apt-get08:35:001
2430999181,6cyclictest11511-21grep10:55:081
24309991812,4cyclictest29700-21switchtime08:45:131
24309991811,5cyclictest22829-21snmpd07:59:541
24309991810,6cyclictest22829-21snmpd09:39:511
2430899189,7cyclictest22829-21snmpd12:38:500
2430899189,7cyclictest1-21systemd10:03:080
2430899189,6cyclictest5487-21aten2_r4power_e10:39:560
24308991811,5cyclictest23730-21awk08:30:080
24308991811,5cyclictest23620-21kworker/u4:109:14:380
24308991811,5cyclictest22829-21snmpd09:06:440
24308991811,4cyclictest2214-21munin-run11:59:380
2430899181,12cyclictest0-21swapper/009:25:020
24308991810,6cyclictest22829-21snmpd12:19:350
24308991810,6cyclictest22829-21snmpd11:47:550
24308991810,6cyclictest22829-21snmpd11:08:420
24308991810,6cyclictest22829-21snmpd09:23:240
24308991810,6cyclictest22829-21snmpd09:18:300
24308991810,6cyclictest22829-21snmpd08:56:560
24308991810,6cyclictest22829-21snmpd07:50:150
24308991810,6cyclictest22829-21snmpd07:43:450
24308991810,6cyclictest22829-21snmpd07:18:140
2430899180,5cyclictest0-21swapper/011:25:090
2430999179,6cyclictest22829-21snmpd11:49:401
2430999177,5cyclictest6581-1kworker/u5:211:04:411
2430999171,4cyclictest22-21ksoftirqd/110:20:001
24309991712,3cyclictest22-21ksoftirqd/108:20:551
24309991711,4cyclictest26224-21kworker/1:110:09:591
24309991711,4cyclictest22829-21snmpd10:36:291
2430999170,8cyclictest22829-21snmpd09:57:531
2430899179,6cyclictest22829-21snmpd08:26:010
2430899178,7cyclictest30083-21seq07:20:520
2430899178,7cyclictest30083-21seq07:20:520
2430899178,6cyclictest30069-21tune2fs10:15:020
2430899177,7cyclictest30572-21seq08:49:580
24308991711,4cyclictest3484-21if_enp2s010:30:110
24308991711,4cyclictest22829-21snmpd11:38:010
24308991711,4cyclictest22829-21snmpd08:46:230
24308991711,4cyclictest20976-21sort08:24:380
2430899171,12cyclictest9616-21apt-get07:55:000
24308991710,5cyclictest22829-21snmpd10:45:200
24308991710,5cyclictest22829-21snmpd09:39:270
24308991710,5cyclictest22829-21snmpd08:12:280
24308991710,5cyclictest22829-21snmpd07:11:020
2430999168,6cyclictest22829-21snmpd12:19:411
2430899169,5cyclictest188-21systemd-journal12:03:170
2430899169,5cyclictest12-21ksoftirqd/011:00:040
2430899168,6cyclictest14584-21swap09:30:160
2430899161,13cyclictest5770-21diskmemload09:58:180
24308991611,3cyclictest22829-21snmpd08:34:490
24308991611,3cyclictest22829-21snmpd07:29:590
24308991610,4cyclictest22829-21snmpd12:07:070
24308991610,4cyclictest22829-21snmpd11:19:330
2430899160,6cyclictest9299-21chrt12:19:590
24308991510,3cyclictest22829-21snmpd10:35:260
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional