You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-27 - 13:51

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Sat Apr 27, 2024 00:45:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
14342997832,9cyclictest131rcu_preempt19:20:100
14342997320,10cyclictest131rcu_preempt20:55:130
1434299691,62cyclictest2697-21ssh23:00:140
14369996442,15cyclictest26866irq/58-eth022:00:123
1434299647,48cyclictest25071-21ssh22:00:120
14342996414,10cyclictest131rcu_preempt00:25:130
14359995912,32cyclictest0-21swapper/223:00:142
1434299563,5cyclictest131rcu_preempt19:30:110
1436999541,27cyclictest0-21swapper/323:00:143
1435999543,32cyclictest12804-21ssh00:20:152
1434299543,7cyclictest131rcu_preempt21:30:120
14342995420,8cyclictest131rcu_preempt19:40:140
14342995420,8cyclictest131rcu_preempt19:40:130
1435999526,16cyclictest27-21ksoftirqd/200:30:122
1435099523,5cyclictest131rcu_preempt20:20:101
1435099503,5cyclictest131rcu_preempt00:05:121
1435999482,40cyclictest12750-21ssh21:35:142
1435099452,31cyclictest9417-21ssh21:15:111
14350994522,17cyclictest0-21swapper/123:45:121
1436999445,25cyclictest26866irq/58-eth021:30:133
1435099421,31cyclictest27351-21ssh21:25:121
1435099402,30cyclictest14845-21ssh23:25:151
1434299372,24cyclictest13815-21ssh22:30:130
14359993611,6cyclictest27-21ksoftirqd/219:20:092
1436999355,24cyclictest26866irq/58-eth022:20:133
1435099352,25cyclictest32506-21diskmemload22:15:101
1434299352,27cyclictest32501-21ssh23:35:120
14350993417,9cyclictest0-21swapper/100:30:121
1436999335,19cyclictest26866irq/58-eth021:40:133
1435999333,19cyclictest22046irq/40-dwc2_hso23:55:132
1435999332,24cyclictest17888-21apt-get23:45:122
1434299331,22cyclictest22807-21ssh22:35:150
1435999321,4cyclictest27-21ksoftirqd/223:05:132
1435999319,14cyclictest0-21swapper/222:00:112
1435999312,20cyclictest18425-21ssh21:20:142
1435999311,27cyclictest0-21swapper/222:15:112
1435099314,19cyclictest1568-21ssh22:05:141
1435099312,23cyclictest10547-21ssh22:10:121
1434299312,20cyclictest21746-21ssh21:40:130
1434299311,24cyclictest12805-21ssh00:20:170
1435999303,10cyclictest0-21swapper/200:25:142
1435999302,19cyclictest31702-21ssh22:40:132
14359993010,15cyclictest0-21swapper/219:30:102
1435099302,18cyclictest21-21ksoftirqd/123:50:101
1435999291,19cyclictest27376-21ssh21:25:132
1435099293,18cyclictest25080-21ssh22:00:121
1434299292,20cyclictest698-21ssh23:15:280
14342992913,10cyclictest3082-21kworker/u8:200:15:100
1435999281,9cyclictest0-21swapper/220:30:102
1435099281,23cyclictest12194-21ssh00:00:121
1434299281,21cyclictest17131-21ssh22:50:130
1436999272,14cyclictest26866irq/58-eth021:35:133
14369992714,4cyclictest26866irq/58-eth022:15:103
1435999276,18cyclictest22046irq/40-dwc2_hso21:15:122
1435999274,15cyclictest22046irq/40-dwc2_hso19:40:142
1435999274,15cyclictest22046irq/40-dwc2_hso19:40:132
1435999274,14cyclictest22046irq/40-dwc2_hso00:15:122
1435999273,17cyclictest22046irq/40-dwc2_hso21:05:142
14350992711,12cyclictest0-21swapper/123:05:131
1434299272,19cyclictest8152-21ssh22:45:100
1434299263,3cyclictest12-21ksoftirqd/000:30:130
1434299263,19cyclictest30671-21ssh21:45:130
1434299263,19cyclictest30671-21ssh21:45:120
1434299261,9cyclictest5657-21cut00:35:210
1436999252,10cyclictest26866irq/58-eth022:50:123
1435999257,15cyclictest6437-21cut21:30:212
1435999257,13cyclictest27-21ksoftirqd/222:10:112
1435999253,13cyclictest27-21ksoftirqd/221:00:102
1435999251,20cyclictest32506-21diskmemload00:00:142
1435999251,18cyclictest13006-21ls23:40:362
1435999251,13cyclictest12132-21awk21:50:382
1435999251,12cyclictest16139-21ssh21:55:102
14359992510,11cyclictest0-21swapper/223:50:122
14359992510,10cyclictest27-21ksoftirqd/223:10:142
1435099252,16cyclictest5888-21ssh23:20:131
1434299252,9cyclictest0-21swapper/020:05:180
1435999246,12cyclictest22046irq/40-dwc2_hso00:10:122
1435999244,11cyclictest21722-21ssh21:40:122
1435999243,16cyclictest22046irq/40-dwc2_hso00:05:092
1435099246,11cyclictest460-21ssh21:10:141
14350992412,9cyclictest2812-21df_inode23:00:151
1434299244,13cyclictest131rcu_preempt21:50:150
14369992311,9cyclictest26866irq/58-eth021:10:153
1435999239,11cyclictest30507-21kthreadcore23:15:212
1435999237,6cyclictest27-21ksoftirqd/223:25:252
1435999235,9cyclictest22046irq/40-dwc2_hso22:50:132
1435999233,10cyclictest8434-21cron20:45:012
1435999232,17cyclictest4931-21ssh22:25:122
1435999231,17cyclictest13820-21ssh22:30:132
1435999231,16cyclictest22046irq/40-dwc2_hso20:00:122
14359992310,4cyclictest31461-21kthreadcore21:45:242
14359992310,4cyclictest31461-21kthreadcore21:45:232
1435099239,10cyclictest0-21swapper/120:00:111
1435099233,13cyclictest0-21swapper/119:15:111
1435099231,16cyclictest18422-21sh21:20:141
1435099231,15cyclictest8849-21cut20:45:171
1434299239,10cyclictest5090-21grep22:25:140
1434299235,5cyclictest131rcu_preempt23:25:140
14342992310,9cyclictest0-21swapper/019:25:110
1436999221,18cyclictest0-21swapper/322:35:343
1435999227,12cyclictest19120-21latency19:45:212
1435999227,11cyclictest27331-21cut22:55:222
1435999224,13cyclictest22046irq/40-dwc2_hso20:05:112
1435999222,10cyclictest3142-21kthreadcore21:10:252
1435999222,10cyclictest29986-21kthreadcore22:20:192
1435999221,9cyclictest31651-21kthreadcore19:25:222
1435999221,9cyclictest10685-21usage_EthcTask23:20:372
1435999221,11cyclictest4146-21cut00:35:152
1435999221,11cyclictest1517-21kthreadcore23:35:202
14359992210,8cyclictest13793-21tail20:50:182
1435099227,4cyclictest131rcu_preempt22:25:181
1435099221,17cyclictest3886-21apt-get20:40:111
14350992210,10cyclictest11449-21cron00:40:021
1435099220,16cyclictest11822-21latency_hist00:00:031
1434299224,14cyclictest26038-21kworker/u8:323:10:150
1434299222,3cyclictest131rcu_preempt23:05:120
14342992212,7cyclictest0-21swapper/020:10:290
1434299221,16cyclictest6336-21cut19:35:130
1434299221,15cyclictest17919-21sh23:45:120
1436999219,8cyclictest26866irq/58-eth000:30:133
14369992114,4cyclictest26866irq/58-eth021:15:403
14369992112,6cyclictest0-21swapper/322:45:163
1435999219,9cyclictest26607-21latency_hist20:30:012
1435999214,11cyclictest0-21swapper/220:45:132
1435999212,16cyclictest18254-21find20:55:122
1435999212,10cyclictest15156-21cut19:10:202
1435999211,9cyclictest8936-21kthreadcore19:35:202
1435999211,9cyclictest17399-21irqcore20:20:182
1435999211,16cyclictest31538-21cpuspeed20:35:152
1435999211,12cyclictest2312-21irqstats22:05:222
1435999211,11cyclictest23049-21fschecks_time22:35:182
1435999211,10cyclictest25970-21sed19:55:202
1435999210,16cyclictest20803-21cut19:50:162
1435099215,11cyclictest0-21swapper/123:40:141
1435099211,18cyclictest12977-21cut00:20:191
1435099211,15cyclictest2809-21ls20:35:321
1435099210,16cyclictest1473-21cut19:30:141
1434299217,3cyclictest131rcu_preempt20:25:190
1434299211,3cyclictest131rcu_preempt23:40:250
1434299211,17cyclictest19570-21timerandwakeup19:45:290
1434299211,17cyclictest14062-21ls22:10:240
1434299211,13cyclictest27817-21ls21:05:000
1434299211,13cyclictest131rcu_preempt21:25:020
1436999202,13cyclictest0-21swapper/300:15:103
14369992011,7cyclictest0-21swapper/321:50:243
14369992011,6cyclictest26866irq/58-eth000:05:103
1436999201,15cyclictest0-21swapper/322:40:213
1435999205,4cyclictest0-21swapper/220:10:112
1435999204,11cyclictest22046irq/40-dwc2_hso19:15:112
1435999203,14cyclictest22046irq/40-dwc2_hso23:30:102
1435999201,9cyclictest14382-21latency_hist19:10:022
14359992011,6cyclictest9208-21ssh22:45:202
1435999201,16cyclictest15969-21timerandwakeup20:15:292
1435099202,16cyclictest26910-21kworker/u8:000:10:231
1435099201,14cyclictest8170-21sh22:45:131
1435099201,14cyclictest26171-21ssh22:55:141
1435099201,10cyclictest30956-21kthreadcore22:20:201
1435099201,10cyclictest25264-21cat21:40:261
1435099200,16cyclictest25579-21cut19:55:181
1434299206,3cyclictest131rcu_preempt21:15:040
1434299205,7cyclictest527-21ssh23:52:090
1434299201,8cyclictest131rcu_preempt20:45:030
1434299201,4cyclictest0-21swapper/023:55:090
14342992013,5cyclictest0-21swapper/022:09:510
14342992011,6cyclictest0-21swapper/019:50:260
1434299201,10cyclictest30227-21sendmail_mailqu20:30:280
1434299200,7cyclictest26154-21apt-get22:55:120
1434299200,16cyclictest32153-21ps20:35:200
1436999199,6cyclictest0-21swapper/322:31:473
1436999193,12cyclictest0-21swapper/300:20:163
1436999192,11cyclictest0-21swapper/323:15:113
14369991912,4cyclictest0-21swapper/322:10:203
14369991910,6cyclictest0-21swapper/323:43:533
14369991910,6cyclictest0-21swapper/321:28:063
14369991910,6cyclictest0-21swapper/300:01:533
1435099192,15cyclictest32506-21diskmemload21:50:151
14350991912,4cyclictest14143-21if_err_eth022:30:171
1435099191,16cyclictest26313-21ssh22:35:271
14350991911,5cyclictest29965-21irqrtprio23:15:191
1435099191,14cyclictest5072-21ssh00:15:171
1435099191,14cyclictest32506-21diskmemload21:45:281
1435099191,14cyclictest32506-21diskmemload21:45:271
1434299192,13cyclictest5906-21sh23:20:150
1434299191,4cyclictest131rcu_preempt00:00:150
1434299191,3cyclictest131rcu_preempt20:15:110
14342991912,5cyclictest0-21swapper/019:15:150
1434299191,10cyclictest0-21swapper/022:41:390
1436999189,6cyclictest26866irq/58-eth023:50:123
1436999187,8cyclictest0-21swapper/323:20:153
1436999182,11cyclictest0-21swapper/321:55:103
1436999182,10cyclictest0-21swapper/322:55:053
14369991811,4cyclictest0-21swapper/323:10:143
14369991810,6cyclictest0-21swapper/323:37:533
14369991810,6cyclictest0-21swapper/323:30:163
14369991810,6cyclictest0-21swapper/323:05:133
14369991810,5cyclictest26866irq/58-eth023:25:133
14369991810,3cyclictest0-21swapper/322:29:083
14369991810,3cyclictest0-21swapper/300:10:263
1435099189,6cyclictest28456-21irqrtprio21:05:201
1435099182,13cyclictest23001-21kthreadcore00:25:231
1435099181,9cyclictest17343-21kthreadcore19:10:231
1435099181,13cyclictest24313-21cut23:30:181
1435099181,13cyclictest15638-21munin-run19:45:011
1435099181,13cyclictest15638-21munin-run19:45:001
1435099181,12cyclictest18263-21apt-get20:55:121
14350991810,6cyclictest7602-21ntp_states21:30:291
14350991810,5cyclictest32537-21cpu23:35:131
14350991810,5cyclictest18202-21kthreadcore21:55:211
1434299189,7cyclictest18114-21cut19:10:300
1434299184,3cyclictest131rcu_preempt21:55:130
1434299181,9cyclictest29262-21irqcore00:10:190
1434299181,8cyclictest26916-21kthreadcore19:55:230
1434299181,2cyclictest131rcu_preempt21:05:140
1434299181,14cyclictest28395-21ssh22:20:110
1434299181,14cyclictest151rcuc/021:25:110
1434299181,14cyclictest12745-21ssh21:35:140
14342991811,4cyclictest0-21swapper/023:30:110
1434299181,13cyclictest450-21ssh21:10:150
1434299181,13cyclictest21630-21apt20:25:020
1436999179,6cyclictest0-21swapper/323:46:053
1436999179,5cyclictest0-21swapper/300:35:203
1436999178,6cyclictest26866irq/58-eth021:20:143
1436999173,9cyclictest26866irq/58-eth021:45:153
1436999173,9cyclictest26866irq/58-eth021:45:143
1436999172,10cyclictest0-21swapper/321:05:143
1436999172,10cyclictest0-21swapper/320:45:133
1436999172,10cyclictest0-21swapper/320:15:123
1436999172,10cyclictest0-21swapper/319:40:053
1436999172,10cyclictest0-21swapper/319:40:043
14369991712,3cyclictest33-21ksoftirqd/300:25:223
14369991711,4cyclictest33-21ksoftirqd/323:55:213
1435099178,6cyclictest24654-21sed23:10:341
1435099173,11cyclictest24551-21grep19:50:311
1435099172,10cyclictest0-21swapper/119:45:121
14350991711,4cyclictest15660-21ntp_192.168.11420:15:251
1435099171,13cyclictest31900-21cut22:40:161
14350991710,5cyclictest21360-21usage_Anybus22:50:341
14350991710,4cyclictest16596-21processes21:35:301
1434299178,6cyclictest15777-21kthreadcore20:50:230
1434299172,10cyclictest0-21swapper/020:45:130
1434299171,13cyclictest19402-21apt-get22:15:100
1434299171,12cyclictest31759-21kthreadcore20:00:200
1434299171,12cyclictest21224-21irqstats00:05:180
1436999168,6cyclictest0-21swapper/322:08:373
1436999162,10cyclictest0-21swapper/320:35:133
1436999162,10cyclictest0-21swapper/319:25:123
1436999161,10cyclictest0-21swapper/320:55:133
1435099168,6cyclictest1-21systemd20:32:211
1435099163,9cyclictest29495-21irqrtprio19:25:181
1435099162,11cyclictest8281-21kthreadcore19:35:181
1435099162,11cyclictest7605-21fschecks_time20:10:131
1435099162,10cyclictest0-21swapper/120:25:141
14350991611,3cyclictest16921-21ntp_offset20:50:281
1435099161,12cyclictest4395-21cut20:05:191
1435099161,12cyclictest25971-21tr21:00:221
1435099161,10cyclictest26321-21kthreadcore19:20:211
1436999151,9cyclictest0-21swapper/321:00:113
1436999151,11cyclictest0-21swapper/319:35:193
1436999151,10cyclictest0-21swapper/320:40:123
1435099151,9cyclictest14421-21latency_hist19:10:021
1436999149,3cyclictest33-21ksoftirqd/319:30:103
1436999141,7cyclictest0-21swapper/319:15:023
1436999141,10cyclictest0-21swapper/319:15:143
1434299141,10cyclictest14371-21latency_hist19:10:010
1436999138,3cyclictest33-21ksoftirqd/320:10:263
1436999131,9cyclictest0-21swapper/320:05:233
1436999131,9cyclictest0-21swapper/320:00:143
1436999131,9cyclictest0-21swapper/319:20:113
1436999131,10cyclictest0-21swapper/320:25:223
1436999128,2cyclictest33-21ksoftirqd/319:55:183
1436999127,3cyclictest0-21swapper/320:50:193
1436999127,3cyclictest0-21swapper/319:45:233
1436999121,9cyclictest0-21swapper/319:55:013
1436999121,6cyclictest0-21swapper/320:20:113
1436999111,7cyclictest0-21swapper/320:30:163
143699991,5cyclictest0-21swapper/319:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional