You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-16 - 13:09

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Tue Apr 16, 2024 00:43:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1104257995949,7cyclictest1224941-21unixbench_singl22:40:351
1104256995228,13cyclictest77550irq/32-end021:30:240
1104256995228,13cyclictest77550irq/32-end021:30:240
1104257994934,12cyclictest1133214-21sshd20:35:241
110425899482,41cyclictest1184292-21ssh21:55:272
1104257994735,8cyclictest1161127-21unin-run21:30:021
1104256994534,8cyclictest1107567-21unin-run19:20:030
110425899444,31cyclictest1197502-21ssh22:10:242
1104256994416,22cyclictest1193010-21ssh22:05:240
1104256994334,5cyclictest77550irq/32-end023:40:300
1104256994334,5cyclictest77550irq/32-end023:40:300
1104256994332,6cyclictest77550irq/32-end023:45:270
1104256994231,8cyclictest77550irq/32-end023:25:270
1104256994218,19cyclictest1232995-21ssh22:50:230
1104257994132,6cyclictest1276586-21grep23:40:191
1104257994132,6cyclictest1276586-21grep23:40:191
1104256994133,5cyclictest1204213-21sshd22:18:240
1104256994017,16cyclictest77550irq/32-end021:45:310
110425799395,29cyclictest1219871-21ssh22:35:261
1104257993920,13cyclictest502-21in:imklog21:05:281
1104256993932,3cyclictest0-21swapper/023:15:250
1104256993923,8cyclictest77550irq/32-end000:30:280
1104258993827,8cyclictest1246401-21needreboot23:05:212
1104258993827,8cyclictest1202487-21unixbench_multi22:15:342
110425799385,29cyclictest1175484-21ssh21:45:261
110425799383,30cyclictest1105374-21sshd19:10:241
1104257993830,5cyclictest1238121-21unixbench_singl22:55:331
1104256993828,6cyclictest1104262-21true19:10:030
1104256993828,6cyclictest1104262-21true19:10:030
1104256993816,18cyclictest0-21swapper/023:10:210
110425899373,25cyclictest1224297-21ssh22:40:262
1104258993729,5cyclictest1298237-21gltestperf00:05:202
1104258993729,5cyclictest1275795-21idleruntime-cro23:40:022
110425799379,7cyclictest171rcu_preempt22:20:271
110425799374,27cyclictest1210947-21ssh22:25:211
110425799373,28cyclictest1202026-21ssh22:15:291
1104257993727,6cyclictest284-21systemd-journal23:55:251
110425699374,13cyclictest1171179-21ssh21:40:280
1104256993722,10cyclictest1153676-21sh21:20:260
110425799366,7cyclictest171rcu_preempt23:05:251
110425799363,7cyclictest171rcu_preempt22:00:251
110425799363,24cyclictest1241946-21ssh23:00:221
1104257993628,5cyclictest1116301-21fschecks_time19:45:121
1104257993625,8cyclictest1114122-21sh19:40:031
1104257993623,10cyclictest1149844-21unixbench_singl21:15:341
1104257993622,11cyclictest1288971-21idleruntime-cro23:55:021
1104256993619,12cyclictest1165939-21aten2.4-expect21:35:170
1104256993615,14cyclictest151ktimers/022:10:240
110425899356,25cyclictest887-21ntpd20:20:252
1104258993527,5cyclictest1324502-21sendmail-msp00:40:022
1104258993527,5cyclictest1312979-21sh00:25:022
1104258993527,5cyclictest1122937-21gltestperf20:05:152
110425899352,27cyclictest1259537-21sshd23:20:242
1104257993527,5cyclictest1316759-21systemd-run00:30:031
1104257993527,5cyclictest1180458-21grep21:50:361
1104257993527,5cyclictest1127233-21sendmail20:20:031
1104257993526,6cyclictest1263328-21fschecks_time23:25:181
1104257993525,7cyclictest1309071-21sendmail00:20:031
1104257993525,7cyclictest1309071-21sendmail00:20:031
110425799352,27cyclictest1268237-21ssh23:30:281
1104257993515,15cyclictest271ktimers/121:55:321
1104256993527,5cyclictest1294309-21gltestperf00:00:200
1104256993527,5cyclictest1125503-21unixbench_singl20:10:350
1104256993527,5cyclictest1115766-21unin-run19:45:020
1104256993526,6cyclictest1231805-21unin-run22:50:030
1104256993526,6cyclictest1111374-21fschecks_time19:30:110
1104256993524,8cyclictest1117381-21munin-plugin-st19:50:020
1104256993519,12cyclictest151ktimers/021:05:280
1104258993427,5cyclictest1320658-21grep00:35:042
1104258993426,3cyclictest0-21swapper/222:05:282
1104258993425,7cyclictest1268109-21needreboot23:30:242
1104258993425,6cyclictest1188291-21cut22:00:192
110425899342,28cyclictest1211007-21rm22:25:232
110425799347,19cyclictest1281469-21ssh23:45:261
1104257993424,7cyclictest1321908-21ssh00:35:271
1104257993424,7cyclictest1134367-21fschecks_time20:40:161
110425799342,26cyclictest1215530-21ssh22:30:251
110425799341,24cyclictest1166845-21ssh21:35:311
110425699346,21cyclictest1179-21munin-node23:55:280
1104256993426,5cyclictest1321419-21gltestperf00:35:190
1104256993426,4cyclictest872-21mta-sts-daemon21:25:260
1104256993424,7cyclictest1209912-21latency_hist22:25:040
1104256993423,8cyclictest1267988-21latency23:30:220
110425699342,28cyclictest1224328-21ssh22:40:270
1104258993325,5cyclictest1255768-21grep23:15:342
1104258993324,6cyclictest1206027-21gltestperf22:20:192
1104258993323,7cyclictest1285795-21ntp_kernel_err23:50:242
1104258993322,8cyclictest1174657-21aten_r2power_fr21:45:152
110425899332,24cyclictest1263893-21ssh23:25:262
1104258993321,9cyclictest1113481-21ntp_kernel_pll_19:35:222
1104258993321,7cyclictest1294684-21ntpq00:00:262
110425799335,7cyclictest171rcu_preempt22:05:261
1104257993320,10cyclictest1232978-21awk22:50:231
1104256993326,4cyclictest1148734-21fschecks_time21:15:180
1104256993324,7cyclictest1314506-21xargs00:25:320
1104256993310,18cyclictest1272574-21ssh23:35:230
110425899325,15cyclictest1302417-21ssh00:10:282
1104258993225,4cyclictest1119161-21latency_hist19:55:042
1104258993219,9cyclictest1106991-21sshd19:15:232
110425799328,18cyclictest435-21mta-sts-daemon21:30:221
110425799328,18cyclictest435-21mta-sts-daemon21:30:221
1104257993224,5cyclictest1125604-21systemd-run20:15:031
1104257993213,11cyclictest271ktimers/120:45:271
110425799321,24cyclictest1317962-21sshd00:30:241
1104256993219,9cyclictest1120043-21ntp_kernel_pll_19:55:190
1104258993124,4cyclictest876-21mta-sts-daemon23:04:032
1104258993123,5cyclictest1166144-21fschecks_count21:35:202
1104258993123,5cyclictest1143651-21diskmemload23:40:282
1104258993123,5cyclictest1143651-21diskmemload23:40:282
1104258993122,6cyclictest1116396-21grep19:45:142
1104258993120,9cyclictest1281032-21gltestperf23:45:192
1104258993119,9cyclictest1129888-21ntp_kernel_err20:25:222
110425899311,21cyclictest1290191-21ntpq23:55:232
1104257993125,3cyclictest1259771-21cat23:20:301
1104257993124,3cyclictest0-21swapper/120:50:181
1104257993123,5cyclictest284-21systemd-journal19:25:151
1104257993122,6cyclictest502-21in:imklog23:10:201
1104257993122,6cyclictest1139245-21fschecks_count20:55:161
1104257993119,7cyclictest1197563-21ntpq22:10:261
110425699312,20cyclictest1219844-21ssh22:35:250
1104256993119,9cyclictest151ktimers/022:30:260
110425899305,17cyclictest876-21mta-sts-daemon21:40:302
110425899303,22cyclictest1215416-21ssh22:30:222
1104258993019,8cyclictest1314211-21ntp_offset00:25:282
1104258993018,9cyclictest1165466-21sh21:35:032
1104258993018,9cyclictest1165466-21sh21:35:032
110425799303,21cyclictest1294752-21ssh00:00:281
110425799303,20cyclictest1228701-21ssh22:45:281
1104256993022,5cyclictest502-21in:imklog19:20:200
1104256993022,5cyclictest1136085-21gltestperf20:45:220
1104256993021,7cyclictest1104255-21cyclictest00:10:030
110425899298,17cyclictest1230838-21sshd22:48:232
1104258992923,3cyclictest0-21swapper/221:25:242
1104258992923,3cyclictest0-21swapper/221:15:202
1104258992921,5cyclictest1152856-21aten2.4-expect21:20:132
1104258992920,5cyclictest1232119-21ssh22:50:132
1104258992919,8cyclictest1179989-21sh21:50:312
110425799296,18cyclictest435-21mta-sts-daemon19:30:191
1104257992922,5cyclictest284-21systemd-journal19:40:091
1104257992922,4cyclictest1118916-21unixbench_multi19:50:261
110425799292,19cyclictest1153668-21ssh21:20:251
1104257992920,6cyclictest1254770-21gltestperf23:15:171
110425799291,23cyclictest1108672-21sshd19:20:241
110425799291,21cyclictest1272711-21ssh23:35:281
110425699295,14cyclictest435-21mta-sts-daemon20:40:250
1104256992918,7cyclictest1184401-21sh21:55:310
110425699291,21cyclictest1306289-21ssh00:15:250
110425699291,21cyclictest1306289-21ssh00:15:250
110425699291,21cyclictest1179893-21sh21:50:270
110425699291,11cyclictest1260178-21grep23:20:330
110425899282,17cyclictest284-21systemd-journal20:35:242
1104258992820,5cyclictest1140232-21unixbench_multi20:55:322
1104258992820,5cyclictest1132109-21unixbench_singl20:30:362
1104258992818,8cyclictest1121501-21latency20:00:142
1104258992817,8cyclictest1143066-21ntp_kernel_pll_21:05:282
110425799287,4cyclictest171rcu_preempt00:10:291
110425799282,19cyclictest1145038-21ssh21:10:311
1104257992821,5cyclictest1104255-21cyclictest21:41:401
1104257992821,4cyclictest1107512-21sshd19:19:231
1104257992818,8cyclictest1129716-21latency20:25:191
110425699282,20cyclictest1144928-21ssh21:10:270
1104256992820,5cyclictest1302950-21unixbench_multi00:10:360
1104256992820,5cyclictest1127234-21idleruntime-cro20:20:030
1104256992814,10cyclictest77550irq/32-end022:55:260
1104256992813,11cyclictest77550irq/32-end022:00:260
110425699281,18cyclictest1241957-21sh23:00:220
1104258992721,3cyclictest0-21swapper/222:35:192
1104258992719,5cyclictest1136036-21fschecks_time20:45:212
1104258992719,5cyclictest1124454-21fschecks_count20:10:182
1104258992716,8cyclictest1250814-21ntp_kernel_err23:10:242
110425899270,24cyclictest1305763-21fschecks_time00:15:182
110425899270,24cyclictest1305763-21fschecks_time00:15:182
110425799277,14cyclictest876-21mta-sts-daemon20:00:191
110425799272,20cyclictest1310374-21ntpq00:20:291
1104257992719,5cyclictest1120646-21cron20:00:021
110425699272,20cyclictest1210947-21ssh22:25:210
1104256992720,4cyclictest1248758-21ssh23:08:320
1104256992719,5cyclictest1122303-21cron20:05:020
1104256992718,6cyclictest1105881-21munin-plugin-st19:15:020
1104256992718,4cyclictest77650irq/33-end023:50:280
1104256992717,7cyclictest1138602-21unixbench_multi20:50:330
110425899262,14cyclictest1237640-21ntpq22:55:292
1104258992619,4cyclictest1120540-21unixbench_multi19:55:282
1104258992618,6cyclictest351ktimers/219:20:212
1104258992617,6cyclictest1127174-21grep20:15:332
1104257992619,5cyclictest502-21in:imklog20:10:031
1104257992618,4cyclictest1132089-21unixbench_singl20:30:361
1104256992620,3cyclictest0-21swapper/000:20:180
1104256992619,5cyclictest0-21swapper/020:10:020
110425899253,15cyclictest872-21mta-sts-daemon19:30:182
1104258992517,5cyclictest1104802-21fschecks_time19:10:122
110425799251,18cyclictest1298747-21ssh00:05:301
1104256992518,4cyclictest876-21mta-sts-daemon21:00:320
110425899243,16cyclictest1145013-21ssh21:10:302
1104258992419,2cyclictest0-21swapper/219:25:292
1104258992414,6cyclictest1135375-21head20:40:322
110425899241,18cyclictest1115243-21sshd19:40:242
1104257992415,6cyclictest1127780-21fschecks_count20:20:151
110425699242,14cyclictest877-21mta-sts-daemon19:25:210
1104256992420,2cyclictest0-21swapper/020:20:210
1104256992417,5cyclictest1129533-21gltestperf20:25:160
110425999233,9cyclictest0-21swapper/321:50:253
110425899232,15cyclictest1138181-21ntp_states20:50:282
1104258992317,3cyclictest0-21swapper/219:10:042
1104258992317,3cyclictest0-21swapper/219:10:042
1104258992315,5cyclictest1140820-21cpuspeed21:00:142
1104257992316,5cyclictest284-21systemd-journal21:00:221
1104256992314,6cyclictest1117936-21fschecks_time19:50:100
110425699231,12cyclictest1131523-21ntpq20:30:260
1104256992215,4cyclictest1113991-21unixbench_multi19:35:300
1104256992214,5cyclictest1140296-21unixbench_singl20:55:320
110425699221,19cyclictest1094025-1kworker/0:1H+kblockd20:35:230
110425999212,11cyclictest0-21swapper/322:25:263
110425999193,11cyclictest0-21swapper/300:35:273
110425999193,10cyclictest0-21swapper/323:40:243
110425999193,10cyclictest0-21swapper/323:40:243
110425999193,10cyclictest0-21swapper/321:35:263
110425999192,11cyclictest0-21swapper/322:15:273
110425999192,11cyclictest0-21swapper/300:30:283
110425999191,13cyclictest0-21swapper/321:25:253
110425999184,7cyclictest0-21swapper/323:20:263
110425999183,9cyclictest0-21swapper/322:20:253
110425999183,9cyclictest0-21swapper/322:05:263
110425999183,9cyclictest0-21swapper/321:25:033
110425999183,9cyclictest0-21swapper/320:35:243
110425999183,9cyclictest0-21swapper/300:00:273
110425999183,8cyclictest0-21swapper/323:20:033
110425999182,9cyclictest0-21swapper/322:30:213
110425999182,10cyclictest0-21swapper/323:50:243
110425799189,6cyclictest284-21systemd-journal19:10:031
110425799189,6cyclictest284-21systemd-journal19:10:031
110425999174,8cyclictest0-21swapper/322:40:263
110425999173,9cyclictest0-21swapper/323:10:143
110425999173,9cyclictest0-21swapper/322:00:243
110425999173,9cyclictest0-21swapper/321:45:313
110425999173,7cyclictest0-21swapper/300:10:263
110425999172,9cyclictest0-21swapper/323:30:283
110425999172,9cyclictest0-21swapper/322:55:143
110425999172,9cyclictest0-21swapper/322:50:233
110425999172,9cyclictest0-21swapper/321:40:263
110425999172,9cyclictest0-21swapper/321:30:233
110425999172,9cyclictest0-21swapper/321:30:233
110425999172,9cyclictest0-21swapper/300:25:253
110425999172,9cyclictest0-21swapper/300:20:243
110425999172,9cyclictest0-21swapper/300:15:263
110425999172,9cyclictest0-21swapper/300:15:263
110425999172,10cyclictest0-21swapper/323:55:233
110425999171,9cyclictest0-21swapper/322:45:183
110425999171,9cyclictest0-21swapper/322:35:223
110425999171,11cyclictest0-21swapper/323:35:303
110425999164,8cyclictest0-21swapper/300:05:293
110425999162,9cyclictest0-21swapper/323:00:223
110425999162,9cyclictest0-21swapper/321:55:263
110425999162,9cyclictest0-21swapper/320:40:163
110425999162,8cyclictest0-21swapper/323:25:243
110425999162,7cyclictest0-21swapper/322:10:133
110425999162,10cyclictest0-21swapper/323:45:273
110425999153,7cyclictest0-21swapper/321:10:033
110425999153,7cyclictest0-21swapper/319:50:223
110425999152,9cyclictest0-21swapper/323:05:123
110425999152,9cyclictest0-21swapper/321:10:213
110425999151,8cyclictest0-21swapper/319:15:233
110425999151,7cyclictest0-21swapper/320:45:273
110425999151,7cyclictest0-21swapper/320:05:203
110425999151,7cyclictest0-21swapper/319:20:183
110425999143,6cyclictest0-21swapper/320:00:173
110425999142,7cyclictest0-21swapper/321:15:243
110425999142,7cyclictest0-21swapper/320:15:033
110425999142,6cyclictest0-21swapper/321:00:273
110425999141,7cyclictest0-21swapper/319:10:183
110425999135,5cyclictest0-21swapper/320:25:213
110425999132,7cyclictest0-21swapper/320:50:243
110425999132,7cyclictest0-21swapper/320:15:263
110425999131,9cyclictest0-21swapper/320:55:163
110425999131,8cyclictest0-21swapper/319:25:263
110425999131,7cyclictest0-21swapper/320:30:243
110425999131,7cyclictest0-21swapper/319:40:193
110425999131,7cyclictest0-21swapper/319:35:233
110425999122,6cyclictest0-21swapper/319:55:123
110425999122,5cyclictest0-21swapper/320:20:153
110425999121,7cyclictest0-21swapper/319:30:123
110425999112,6cyclictest0-21swapper/319:45:143
11042599991,5cyclictest0-21swapper/319:10:033
11042599991,5cyclictest0-21swapper/319:10:033
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional