You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-09-08 - 18:50

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Sun Sep 08, 2024 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
10461210374,9sleep10-21swapper/107:08:401
10558210086,9sleep30-21swapper/307:09:553
835328167,9sleep00-21swapper/007:05:100
1036027565,6sleep20-21swapper/207:07:372
72752690,2sleep22228-21kworker/2:008:09:542
1065699300,2cyclictest15055-21cpu12:05:141
1065699300,1cyclictest0-21swapper/109:45:151
1065699293,25cyclictest3803-21diskmemload12:27:371
1065699290,28cyclictest0-21swapper/112:20:261
1065699282,19cyclictest3803-21diskmemload10:50:021
1065699280,3cyclictest16992-21idleruntime-cro08:30:001
1065699280,1cyclictest0-21swapper/109:34:151
1065699270,1cyclictest0-21swapper/110:20:131
1065699264,21cyclictest3803-21diskmemload10:00:351
1065699262,23cyclictest0-21swapper/112:03:131
1065699261,24cyclictest3803-21diskmemload10:38:071
1065699261,1cyclictest3803-21diskmemload09:28:191
1065699260,25cyclictest0-21swapper/109:42:551
1065699260,25cyclictest0-21swapper/109:24:491
1066299256,3cyclictest39-21ksoftirqd/312:39:103
1065699253,20cyclictest26594-21cron08:50:001
1065699251,23cyclictest3803-21diskmemload10:41:261
1065699250,6cyclictest3803-21diskmemload11:56:111
1065699250,24cyclictest0-21swapper/111:27:481
1066299243,3cyclictest381rcuc/311:14:513
1065699242,21cyclictest3803-21diskmemload09:35:361
1065699240,1cyclictest0-21swapper/112:19:141
1065499240,2cyclictest0-21swapper/010:07:370
1065699233,20cyclictest0-21swapper/109:56:301
1065699233,18cyclictest3803-21diskmemload11:36:321
1065699232,20cyclictest3803-21diskmemload11:17:321
1065699232,20cyclictest0-21swapper/110:55:191
1065699232,19cyclictest29683-21unixbench_singl07:45:281
1065699231,21cyclictest0-21swapper/110:15:161
1065699230,22cyclictest0-21swapper/111:22:451
1065699230,22cyclictest0-21swapper/107:35:251
1065699230,1cyclictest0-21swapper/110:46:481
1065699230,1cyclictest0-21swapper/110:08:191
1065699230,18cyclictest0-21swapper/109:10:221
1065699221,17cyclictest3803-21diskmemload11:32:071
1065699220,3cyclictest0-21swapper/110:27:061
1065699220,20cyclictest0-21swapper/111:45:201
1065699220,19cyclictest1879-21ssh12:35:251
1065999210,1cyclictest0-21swapper/209:17:502
1065699213,18cyclictest3803-21diskmemload11:41:431
1065699212,17cyclictest3803-21diskmemload10:31:391
1065699210,3cyclictest12146-21unixbench_multi09:15:271
1065699210,19cyclictest0-21swapper/111:02:321
1065699210,15cyclictest0-21swapper/108:00:181
10654992120,1cyclictest0-21swapper/011:10:140
1066299204,5cyclictest4256-21ssh11:58:483
10662992019,1cyclictest0-21swapper/307:50:133
10662992015,5cyclictest39-21ksoftirqd/307:40:153
1066299200,19cyclictest0-21swapper/307:55:133
1065699201,16cyclictest0-21swapper/108:30:211
1065699200,2cyclictest23379-21ssh11:50:221
1065699200,2cyclictest0-21swapper/111:05:421
1065699200,16cyclictest0-21swapper/112:31:221
1065499206,8cyclictest18938-21awk10:50:000
1065499200,20cyclictest0-21swapper/009:22:560
1066299190,19cyclictest0-21swapper/311:25:143
10659991916,2cyclictest1054-21head07:55:172
1065699191,12cyclictest23167-21grep08:40:211
1065699190,16cyclictest3803-21diskmemload10:11:581
1065499193,7cyclictest224-21systemd-journal11:05:000
1065699180,4cyclictest3803-21diskmemload12:12:011
1066299178,9cyclictest39-21ksoftirqd/311:41:003
10662991715,2cyclictest39-21ksoftirqd/309:29:333
1066299171,14cyclictest23694-21ssh09:47:153
1066299171,12cyclictest23077-21ssh12:10:103
1065999173,7cyclictest0-21swapper/207:30:142
10659991716,1cyclictest8866-21kworker/2:210:00:392
1065999170,17cyclictest872-21kworker/2:010:45:582
1065999170,17cyclictest19251-21kworker/2:109:32:382
1065699171,15cyclictest3803-21diskmemload09:52:221
1065699170,1cyclictest0-21swapper/111:10:121
1065699170,1cyclictest0-21swapper/109:00:201
1065699170,1cyclictest0-21swapper/109:00:191
1065699170,1cyclictest0-21swapper/107:15:161
1065699170,15cyclictest0-21swapper/108:10:201
10654991715,0cyclictest0-21swapper/011:18:310
1066299169,6cyclictest0-21swapper/311:00:003
1066299167,5cyclictest20097-21tune2fs10:50:163
10662991615,1cyclictest3803-21diskmemload10:36:233
10662991614,1cyclictest0-21swapper/309:24:503
10662991610,2cyclictest12156-21cron08:19:593
10659991612,3cyclictest0-21swapper/210:09:582
1065999160,16cyclictest3803-21diskmemload11:29:332
1065999160,14cyclictest0-21swapper/211:40:142
1065699163,12cyclictest20788-21ntp_states08:35:221
1065699161,1cyclictest0-21swapper/108:50:141
1065699161,13cyclictest30814-21timerwakeupswit08:55:261
1065699161,13cyclictest2392-21gltestperf09:05:201
1065699160,2cyclictest0-21swapper/108:20:111
1065499167,5cyclictest23139-21apt-get07:35:120
1065499160,2cyclictest0-21swapper/012:05:100
1065499160,1cyclictest0-21swapper/011:38:410
1065499160,16cyclictest0-21swapper/012:28:130
1066299159,5cyclictest9663-21sh12:20:313
1066299158,7cyclictest3803-21diskmemload11:17:193
1066299158,3cyclictest381rcuc/310:46:483
1066299153,12cyclictest0-21swapper/310:16:043
1066299151,1cyclictest0-21swapper/312:15:123
10662991510,3cyclictest16989-21perf08:30:003
1066299150,13cyclictest0-21swapper/310:20:133
1065999154,10cyclictest24230-21sh09:47:452
1065999150,2cyclictest0-21swapper/212:18:312
1065699150,3cyclictest5193-21kworker/u8:107:54:591
1065699150,1cyclictest0-21swapper/107:20:251
1065699150,1cyclictest0-21swapper/107:10:121
1065499151,8cyclictest19375-21munin-run08:35:000
10654991510,3cyclictest25667-21cut07:40:150
303192140,0chrt0-21swapper/012:33:400
1066299144,6cyclictest3803-21diskmemload11:04:183
10662991414,0cyclictest0-21swapper/310:05:133
1066299141,2cyclictest0-21swapper/310:41:543
10659991413,1cyclictest0-21swapper/212:00:132
1065999141,2cyclictest21081-21ssh09:23:322
10659991412,1cyclictest4357-21aten2_r7power_e09:10:162
10659991410,2cyclictest21706-21uniq11:30:172
10659991410,2cyclictest20393-21cat07:30:002
1065999140,14cyclictest5741-21kworker/u8:110:42:142
1065699148,5cyclictest20648-21cut07:30:141
1065699141,12cyclictest0-21swapper/108:05:171
1065699140,12cyclictest0-21swapper/107:55:121
10654991410,3cyclictest13847-21wc11:25:190
10654991410,2cyclictest32171-21grep10:15:170
1065499140,0cyclictest0-21swapper/012:36:120
1066299139,3cyclictest7744-21snmpd07:22:593
1066299139,3cyclictest3803-21diskmemload10:31:173
1066299139,3cyclictest26598-21sh08:50:003
1066299135,4cyclictest0-21swapper/309:55:153
1066299134,7cyclictest0-21swapper/312:05:143
10662991312,1cyclictest39-21ksoftirqd/312:30:363
10662991311,2cyclictest39-21ksoftirqd/310:00:163
10662991310,2cyclictest4413-21grep08:00:323
10662991310,2cyclictest21281-21grep07:30:173
10662991310,2cyclictest19367-21idleruntime-cro08:34:593
10662991310,1cyclictest381rcuc/309:34:153
1065999139,3cyclictest29618-21sed10:55:262
1065999139,3cyclictest24518-21cut10:10:152
1065999137,5cyclictest31011-21perf12:15:002
10659991310,2cyclictest4652-21tr11:20:142
10659991310,2cyclictest25033-21grep12:30:172
10659991310,2cyclictest24195-21cron08:45:002
10659991310,2cyclictest23677-21dump-pmu-power12:30:012
1065999130,13cyclictest0-21swapper/211:00:142
1065999130,12cyclictest5823-21grep09:35:192
1065999130,12cyclictest14401-21cut07:15:192
1065699137,5cyclictest19904-21chrt07:26:021
1065499139,3cyclictest22355-21grep11:50:140
1065499139,3cyclictest12587-21cut11:05:160
1065499135,4cyclictest224-21systemd-journal12:20:000
1065499134,5cyclictest31407-21munin-run09:00:010
1065499133,9cyclictest5313-21/usr/sbin/munin11:40:290
10654991310,2cyclictest31977-21grep07:50:270
10654991310,2cyclictest13554-21grep07:15:130
10654991310,2cyclictest11499-21cut07:10:170
1065499130,1cyclictest0-21swapper/010:42:140
1065499130,12cyclictest21223-21wc10:50:260
1066299129,2cyclictest5421-21grep09:35:173
1066299129,2cyclictest381rcuc/308:05:183
1066299129,2cyclictest29018-21turbostat.cron08:55:013
1066299129,2cyclictest28416-21cut09:50:213
1066299129,2cyclictest22922-21tail11:50:183
1066299129,1cyclictest381rcuc/311:33:023
1066299128,3cyclictest29960-21seq10:13:593
1066299127,3cyclictest381rcuc/309:40:253
1066299125,7cyclictest0-21swapper/311:05:413
1066299124,7cyclictest13301-21apt-get07:15:133
1066299123,6cyclictest39-21ksoftirqd/310:25:403
1066299123,4cyclictest6101-21apt-key12:00:013
1066299122,9cyclictest7564-21apt-get08:10:143
1066299122,2cyclictest0-21swapper/309:17:453
1066299122,1cyclictest224-21systemd-journal09:00:003
10662991210,1cyclictest4615-21grep11:20:133
1066299120,1cyclictest0-21swapper/307:36:273
1065999129,2cyclictest974-21sed12:35:182
1065999129,2cyclictest31724-21tr10:15:152
1065999129,2cyclictest31221-21wc11:35:252
1065999129,2cyclictest30377-21sed11:15:272
1065999129,2cyclictest30148-21cat07:50:002
1065999129,2cyclictest28943-21grep09:50:262
1065999129,2cyclictest23340-21tr09:25:152
1065999129,2cyclictest19717-21tr10:50:142
1065999129,2cyclictest18025-21head08:30:242
1065999128,3cyclictest8218-21sed10:20:262
1065999128,3cyclictest2155-21ssh10:39:272
1065999128,3cyclictest20722-21grep10:30:002
1065999128,3cyclictest16977-21apt08:30:002
1065999122,3cyclictest224-21systemd-journal09:45:002
1065999120,11cyclictest22534-21tr10:30:192
1065999120,11cyclictest0-21swapper/207:24:572
1065699126,5cyclictest11953-21taskset08:17:461
1065499129,2cyclictest9477-21head12:20:250
1065499129,2cyclictest5399-21tr09:35:160
1065499129,2cyclictest4414-21ssh11:58:560
1065499129,2cyclictest28881-21grep09:50:260
1065499129,2cyclictest24697-21grep08:45:120
1065499129,2cyclictest2449-21cron07:59:590
1065499129,2cyclictest13677-21grep08:20:200
1065499128,3cyclictest21581-21grep09:45:250
1065499128,3cyclictest13-21ksoftirqd/009:41:450
1065499128,2cyclictest28615-21grep07:45:180
1065499128,2cyclictest24199-21cron08:45:000
1065499122,2cyclictest9436-21apache211:48:210
10654991210,1cyclictest10604-21grep09:15:160
1065499120,11cyclictest28326-21ssh10:55:160
1065499120,11cyclictest12200-21sleep08:20:000
1066299119,1cyclictest14427-21tune2fs11:45:153
1066299119,1cyclictest11172-21cut07:10:143
1066299118,2cyclictest16672-21grep12:25:183
1065999118,2cyclictest5285-21taskset11:59:382
1065999118,2cyclictest5020-21ssh09:56:282
1065999118,2cyclictest3521-21head08:00:222
1065999118,2cyclictest24024-21grep11:50:272
1065999118,2cyclictest19918-21grep08:35:172
1065999118,2cyclictest13693-21apache_processe11:45:102
1065999118,2cyclictest12720-21cut08:20:132
1065999118,2cyclictest11192-21cstates07:10:142
1065999117,3cyclictest12155-21cron08:20:002
1065999117,2cyclictest10712-21ssh12:21:192
1065999116,4cyclictest14276-21ssh11:05:442
1065999113,7cyclictest21710-21/usr/sbin/munin11:10:232
1065999110,10cyclictest17974-21ssh12:06:182
1065699110,2cyclictest224-21systemd-journal07:40:151
1065499119,1cyclictest22267-21vmstat07:30:280
1065499118,2cyclictest8209-21cut08:10:160
1065499118,2cyclictest7049-21sort10:20:160
1065499118,2cyclictest5368-21uname11:20:190
1065499118,2cyclictest5033-21wc08:05:000
1065499118,2cyclictest4764-21rm09:56:160
1065499118,2cyclictest29553-21ssh10:35:210
1065499118,2cyclictest29121-21ssh10:13:080
1065499118,2cyclictest27776-21ssh10:34:520
1065499118,2cyclictest23241-21ssh11:30:380
1065499118,2cyclictest13855-21bc10:25:120
1065499118,2cyclictest11771-21ssh10:00:390
1065499118,2cyclictest10407-21ssh12:01:540
1065499117,3cyclictest19598-21wc07:25:240
1065499117,1cyclictest34402sleep009:05:390
1065499114,3cyclictest224-21systemd-journal08:39:590
1065499110,10cyclictest27867-21ssh09:28:090
1065499110,0cyclictest0-21swapper/009:30:440
1066299108,2cyclictest30178-21ssh11:35:163
1066299108,1cyclictest4322-21rm09:10:163
1066299108,1cyclictest19780-21egrep08:35:163
1066299107,2cyclictest22459-21expr08:40:163
1066299101,6cyclictest39-21ksoftirqd/309:05:193
1066299101,5cyclictest18184-21aten2_r7power_e07:25:133
1065999108,1cyclictest9319-21taskset08:10:292
1065999108,1cyclictest36162sleep209:07:562
1065999108,1cyclictest292302sleep208:55:032
1065999108,1cyclictest285932sleep208:50:262
1065999108,1cyclictest274382sleep207:42:092
1065999108,1cyclictest251022sleep207:38:102
1065999106,3cyclictest30971-21grep07:50:172
1065999106,3cyclictest25181-21grep08:45:142
1065999100,9cyclictest1032-21taskset09:00:482
1065999100,9cyclictest1032-21taskset09:00:472
1065499108,1cyclictest27888-21wc08:50:180
1065499108,1cyclictest22893-21ssh12:10:010
1065499107,2cyclictest31690-21seq09:00:140
1065499107,2cyclictest31690-21seq09:00:130
1065499104,2cyclictest224-21systemd-journal07:25:010
1065499100,2cyclictest4589-21cut09:10:160
106629997,1cyclictest31638-21sshd09:00:023
106629997,1cyclictest31638-21sshd09:00:013
106629996,2cyclictest28210-21grep07:45:163
106629991,1cyclictest141rcu_preempt08:20:053
106549998,1cyclictest5660-21sed08:05:180
106549990,5cyclictest16987-21munin-plugin-st08:30:000
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional