You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-05-02 - 23:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot6s.osadl.org (updated Fri May 02, 2025 12:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
130072920,5sleep226-21rcuc/206:01:582
13900999039,9cyclictest11215-21sshd07:04:511
227799860,4rtkit-daemon2276-21rtkit-daemon07:05:252
45222620,5sleep3361ktimersoftd/310:12:083
101772613,51sleep00-21swapper/007:02:030
13900995936,7cyclictest11313-21gltestperf05:57:051
13897995925,12cyclictest0-21swapper/008:02:120
1390299583,49cyclictest0-21swapper/310:21:353
300952575,17sleep20-21swapper/209:58:382
13901995742,9cyclictest12421-21gltestperf07:07:012
1390099573,46cyclictest13609-21crond09:26:361
13902995638,11cyclictest27468-21tr06:31:573
1390299563,36cyclictest0-21swapper/306:22:083
1390299553,46cyclictest0-21swapper/309:37:063
13902995529,10cyclictest361ktimersoftd/305:17:163
13897995530,13cyclictest0-21swapper/008:28:150
314532542,9sleep30-21swapper/308:54:553
227799540,5rtkit-daemon2276-21rtkit-daemon05:12:521
13902995439,9cyclictest26243-21tr07:36:573
13901995438,10cyclictest2205-21grep09:01:522
1389799547,29cyclictest0-21swapper/006:52:100
1389799547,29cyclictest0-21swapper/006:52:100
13897995429,13cyclictest0-21swapper/009:07:030
13897995429,12cyclictest0-21swapper/007:52:010
13897995429,12cyclictest0-21swapper/006:41:590
105132543,43sleep00-21swapper/009:17:090
227799530,5rtkit-daemon2276-21rtkit-daemon06:16:571
227799530,5rtkit-daemon2276-21rtkit-daemon06:11:051
227799530,5rtkit-daemon2276-21rtkit-daemon06:11:051
13902995329,9cyclictest16360-21telnet09:31:483
13902995329,10cyclictest30818-21munin-node05:32:173
13902995323,12cyclictest0-21swapper/308:22:003
13901995336,9cyclictest5009-21unixbench_multi05:42:212
13901995333,13cyclictest8548-21grep06:57:152
1390099532,44cyclictest3301-21munin-node05:42:141
1389799535,30cyclictest0-21swapper/007:07:170
1389799534,42cyclictest0-21swapper/005:51:440
13897995325,7cyclictest389-21latency_hist07:51:410
13897995325,12cyclictest0-21swapper/009:13:440
13897995325,12cyclictest0-21swapper/007:57:000
13897995324,13cyclictest0-21swapper/008:11:520
227799520,14rtkit-daemon2276-21rtkit-daemon06:12:051
181162522,43sleep10-21swapper/107:17:191
151422527,7sleep00-21swapper/004:57:120
13902995231,8cyclictest8079-21munin-node05:52:143
13902995220,13cyclictest0-21swapper/304:56:483
13902995219,13cyclictest0-21swapper/306:51:423
1390199524,25cyclictest0-21swapper/208:12:012
13901995237,8cyclictest19773-21crond06:16:432
13901995235,9cyclictest7023-21grep09:11:542
13901995233,10cyclictest25671-21crond07:36:402
13901995233,10cyclictest25671-21crond07:36:402
1389799525,41cyclictest0-21swapper/005:02:050
1389799524,41cyclictest13614-21latency_hist09:26:360
1389799524,41cyclictest0-21swapper/010:01:340
13897995230,12cyclictest0-21swapper/005:17:010
13897995230,11cyclictest0-21swapper/007:37:100
13897995227,12cyclictest0-21swapper/010:21:340
13897995227,10cyclictest17790-21rt-features07:17:140
13897995226,9cyclictest7955-21munin-node08:07:030
13897995224,9cyclictest18384-21latency_hist09:36:360
13897995223,13cyclictest0-21swapper/006:02:010
13897995223,12cyclictest0-21swapper/009:51:450
63452513,11sleep20-21swapper/208:02:022
301122517,38sleep00-21swapper/007:42:210
230072512,43sleep10-21swapper/109:44:471
230072512,43sleep10-21swapper/109:44:471
227799510,5rtkit-daemon2276-21rtkit-daemon06:47:041
227799510,5rtkit-daemon2276-21rtkit-daemon06:22:051
217602512,44sleep00-21swapper/008:33:120
13902995137,8cyclictest14918-21hwlatdetect07:12:033
1390299513,42cyclictest19449-21yum-updatesd-he06:12:433
13902995129,9cyclictest18888-21munin-node05:07:013
13902995128,9cyclictest30687-21munin-node07:47:103
13902995128,9cyclictest15401-21telnet06:07:003
13902995128,9cyclictest15401-21telnet06:07:003
13902995128,7cyclictest361ktimersoftd/308:32:023
1390299512,21cyclictest0-21swapper/309:54:563
13902995119,13cyclictest0-21swapper/309:11:383
13901995135,8cyclictest8822-21grep05:52:032
13901995135,10cyclictest31001-21tr07:46:552
13901995134,10cyclictest26798-21crond08:46:372
13901995130,13cyclictest21631-21unixbench_singl08:32:122
13900995129,11cyclictest0-21swapper/105:01:581
13897995129,12cyclictest0-21swapper/005:36:540
13897995128,17cyclictest0-21swapper/009:37:110
13897995127,13cyclictest0-21swapper/005:42:020
13897995124,12cyclictest0-21swapper/006:12:000
313462504,9sleep00-21swapper/008:53:310
241522503,40sleep00-21swapper/008:38:160
222712503,41sleep322295-21iostat_ios05:12:103
13902995033,12cyclictest0-21swapper/306:27:083
13902995028,6cyclictest361ktimersoftd/306:57:003
13902995027,7cyclictest361ktimersoftd/307:16:563
13902995026,8cyclictest361ktimersoftd/307:52:093
13902995025,11cyclictest12730-21munin-node08:17:083
13902995025,11cyclictest12730-21munin-node08:17:083
13902995024,9cyclictest19815-21munin-node07:22:063
13902995022,12cyclictest0-21swapper/308:51:383
13901995035,8cyclictest22932-21df06:22:012
13901995031,11cyclictest13445-21tune2fs08:17:002
13901995031,11cyclictest13445-21tune2fs08:17:002
1390099503,42cyclictest17262-21crond08:26:381
1390099503,24cyclictest0-21swapper/107:12:411
13900995013,30cyclictest0-21swapper/106:56:441
1389799504,40cyclictest0-21swapper/008:22:010
13897995023,12cyclictest0-21swapper/010:02:040
13897995023,12cyclictest0-21swapper/009:51:280
13897995023,12cyclictest0-21swapper/005:56:440
13897995023,12cyclictest0-21swapper/005:12:180
13897995022,12cyclictest0-21swapper/008:41:570
13897995022,12cyclictest0-21swapper/005:56:580
13897995022,10cyclictest30086-21grep05:27:170
1389799502,43cyclictest0-21swapper/005:32:120
136372504,6sleep00-21swapper/004:53:500
123992503,41sleep10-21swapper/108:15:351
227799490,4rtkit-daemon2276-21rtkit-daemon06:31:571
13902994936,8cyclictest1414-21gltestperf10:06:543
13902994932,10cyclictest28297-21aten_r1power_po09:56:463
1390299493,39cyclictest30721-21telnet10:01:483
1390299493,26cyclictest0-21swapper/309:31:363
13902994927,9cyclictest1138-21snmpd09:23:143
13902994924,9cyclictest3361-21telnet05:42:003
13902994919,13cyclictest0-21swapper/306:36:563
13902994918,12cyclictest0-21swapper/308:06:403
1390199494,36cyclictest0-21swapper/209:11:382
13901994934,9cyclictest30645-21needreboot08:52:042
1390099493,41cyclictest0-21swapper/107:42:031
13900994921,12cyclictest0-21swapper/108:01:551
1389799493,41cyclictest0-21swapper/007:22:590
13897994928,12cyclictest0-21swapper/007:12:320
13897994924,9cyclictest7819-21munin-node10:21:490
13897994923,9cyclictest1905-21head09:01:500
13897994922,11cyclictest0-21swapper/006:46:570
13897994921,14cyclictest0-21swapper/006:07:010
13897994921,14cyclictest0-21swapper/006:07:010
13897994921,13cyclictest0-21swapper/005:07:090
13897994920,12cyclictest0-21swapper/007:27:040
5912483,37sleep30-21swapper/308:57:053
278522483,39sleep10-21swapper/107:39:001
276182482,41sleep10-21swapper/109:52:291
254642482,10sleep10-21swapper/109:50:311
246432484,38sleep20-21swapper/205:17:092
227799480,5rtkit-daemon2276-21rtkit-daemon06:27:191
227799480,4rtkit-daemon2276-21rtkit-daemon05:47:131
173032482,9sleep20-21swapper/206:10:422
173032482,9sleep20-21swapper/206:10:422
1390299483,39cyclictest24416-21latency_hist08:41:383
13902994825,10cyclictest7955-21munin-node08:07:163
13902994824,8cyclictest1091-21lldpd09:44:573
13902994824,8cyclictest1091-21lldpd09:44:573
13902994823,11cyclictest0-21swapper/307:27:153
13902994821,10cyclictest4440-21munin-node06:52:073
13902994821,10cyclictest4440-21munin-node06:52:073
1390299482,40cyclictest12315-21sshd05:57:543
13901994833,9cyclictest32577-21head05:32:232
13901994832,9cyclictest15763-21grep07:12:162
13901994832,10cyclictest1386-21fschecks_time07:51:592
13901994831,10cyclictest3900-21tune2fs07:56:592
13900994826,11cyclictest0-21swapper/109:07:071
13900994826,11cyclictest0-21swapper/105:27:181
13900994825,11cyclictest0-21swapper/105:41:461
13900994825,10cyclictest0-21swapper/110:17:001
13900994821,11cyclictest0-21swapper/106:51:561
13900994821,11cyclictest0-21swapper/106:51:561
1389799486,27cyclictest0-21swapper/005:22:030
1389799486,27cyclictest0-21swapper/005:22:030
13897994827,12cyclictest0-21swapper/006:27:100
13897994823,9cyclictest514-21sh10:06:450
13897994823,12cyclictest0-21swapper/009:41:560
13897994823,12cyclictest0-21swapper/009:41:560
13897994823,11cyclictest0-21swapper/008:16:590
13897994823,11cyclictest0-21swapper/008:16:590
13897994823,10cyclictest0-21swapper/006:57:090
13897994821,13cyclictest0-21swapper/010:15:260
13897994821,13cyclictest0-21swapper/006:22:020
13897994821,12cyclictest0-21swapper/006:36:540
13002483,9sleep10-21swapper/108:58:161
60492473,38sleep20-21swapper/210:16:522
43032472,38sleep10-21swapper/110:12:031
164862472,38sleep20-21swapper/205:01:492
161082472,9sleep30-21swapper/304:59:493
13902994732,9cyclictest9943-21gltestperf09:16:583
1390299473,26cyclictest0-21swapper/307:32:203
1390299473,26cyclictest0-21swapper/307:32:203
1390299473,26cyclictest0-21swapper/306:17:123
13902994726,8cyclictest361ktimersoftd/305:02:113
13902994725,8cyclictest10228-21iostat07:02:033
13902994724,8cyclictest30446-21crond07:46:403
13902994723,10cyclictest3190-21munin-node07:57:123
13902994720,10cyclictest23387-21munin-node09:46:473
1390199475,34cyclictest0-21swapper/208:36:542
1390199475,15cyclictest0-21swapper/210:06:522
1390199475,14cyclictest0-21swapper/205:47:072
1390199475,14cyclictest0-21swapper/205:27:032
1390199475,13cyclictest0-21swapper/207:22:102
1390199474,13cyclictest0-21swapper/209:31:542
1390199474,13cyclictest0-21swapper/206:52:072
1390199474,13cyclictest0-21swapper/206:52:072
13901994732,9cyclictest32164-21grep08:56:522
1390199473,36cyclictest0-21swapper/205:07:202
13901994727,13cyclictest28915-21grep06:32:162
13901994714,27cyclictest281ktimersoftd/209:27:112
13900994725,11cyclictest0-21swapper/110:21:511
1389799474,38cyclictest0-21swapper/009:27:080
13897994726,12cyclictest0-21swapper/007:31:550
13897994726,12cyclictest0-21swapper/007:31:550
13897994722,10cyclictest27165-21munin-node06:32:090
13897994718,13cyclictest0-21swapper/008:46:550
99942462,39sleep10-21swapper/105:53:481
290772462,9sleep20-21swapper/208:50:032
279172463,9sleep20-21swapper/209:56:252
266322462,9sleep30-21swapper/308:44:313
232032465,9sleep10-21swapper/107:30:371
227799460,4rtkit-daemon2276-21rtkit-daemon06:36:571
220022462,38sleep10-21swapper/108:36:231
1390299463,37cyclictest0-21swapper/305:22:013
1390299463,37cyclictest0-21swapper/305:22:013
1390299463,12cyclictest29728-21open_inodes05:27:123
1390199466,33cyclictest0-21swapper/208:21:532
1390199465,13cyclictest0-21swapper/208:27:132
1390199464,35cyclictest0-21swapper/207:26:582
13901994628,12cyclictest3757-21grep06:47:182
13901994628,11cyclictest14483-21grep04:57:032
13901994627,12cyclictest9174-21head09:16:512
1390099463,38cyclictest0-21swapper/110:06:581
1390099463,37cyclictest0-21swapper/108:16:571
1390099463,37cyclictest0-21swapper/108:16:571
13900994624,11cyclictest0-21swapper/108:46:571
13900994624,11cyclictest0-21swapper/107:06:561
13900994624,10cyclictest0-21swapper/109:37:001
13900994623,10cyclictest0-21swapper/105:11:461
13900994622,11cyclictest0-21swapper/109:17:111
13900994618,13cyclictest0-21swapper/109:02:081
13900994617,13cyclictest0-21swapper/107:22:121
13900994617,11cyclictest0-21swapper/109:26:511
13897994619,13cyclictest0-21swapper/009:00:460
135202463,9sleep30-21swapper/307:08:473
6172452,37sleep10-21swapper/107:51:451
50472452,37sleep10-21swapper/107:57:581
280362452,38sleep10-21swapper/105:24:501
280362452,38sleep10-21swapper/105:24:501
193572452,9sleep30-21swapper/308:27:553
13902994520,9cyclictest0-21swapper/306:02:003
1390199456,32cyclictest0-21swapper/206:42:142
1390199454,12cyclictest0-21swapper/210:21:522
13901994532,7cyclictest12191-21unixbench_multi05:57:192
13901994530,9cyclictest25641-21gltestperf06:27:062
13901994530,10cyclictest21687-21unixbench-2d06:17:172
13901994526,13cyclictest0-21swapper/210:11:472
13901994526,13cyclictest0-21swapper/205:36:592
13901994521,15cyclictest0-21swapper/207:17:052
13900994530,11cyclictest0-21swapper/109:57:011
13900994522,11cyclictest0-21swapper/106:42:081
13900994520,13cyclictest0-21swapper/110:01:491
13900994520,12cyclictest0-21swapper/109:31:481
13900994519,10cyclictest32260-21taskset05:32:191
13900994519,10cyclictest1263-21runrttasks08:26:541
1389799456,33cyclictest0-21swapper/006:17:090
135232452,9sleep20-21swapper/209:25:332
1390299443,32cyclictest10399-21telnet08:11:523
13902994426,13cyclictest0-21swapper/305:47:163
1390299442,37cyclictest7819-21munin-node10:22:083
1390199445,31cyclictest0-21swapper/209:46:492
1390199444,30cyclictest0-21swapper/207:41:412
1390199444,13cyclictest0-21swapper/206:41:432
13901994426,11cyclictest19822-21tr09:37:012
13901994425,12cyclictest1263-21runrttasks05:22:062
13901994425,12cyclictest1263-21runrttasks05:22:062
13901994424,12cyclictest8532-21entropy08:06:572
1390099443,37cyclictest0-21swapper/105:00:581
1390099443,36cyclictest0-21swapper/108:51:501
13900994429,9cyclictest23863-21tr07:31:581
13900994429,9cyclictest23863-21tr07:31:581
13900994420,12cyclictest0-21swapper/107:48:361
1390099442,37cyclictest0-21swapper/106:04:211
28602432,9sleep30-21swapper/305:39:113
1390299433,33cyclictest501-21iostat_ios06:42:073
1390299433,24cyclictest0-21swapper/309:12:093
1390199435,33cyclictest0-21swapper/207:42:062
13901994322,13cyclictest1263-21runrttasks09:41:512
13901994322,13cyclictest1263-21runrttasks09:41:512
13901994322,12cyclictest32316-21grep10:02:092
13901994319,15cyclictest0-21swapper/205:12:092
1390099432,34cyclictest0-21swapper/108:06:531
13900994314,13cyclictest0-21swapper/109:12:111
13900994314,13cyclictest0-21swapper/105:16:581
13900994314,12cyclictest0-21swapper/108:36:511
1390299423,26cyclictest0-21swapper/309:02:483
13900994219,8cyclictest26813-21idleruntime-cro08:46:371
1390099403,32cyclictest0-21swapper/104:56:481
1390199382,22cyclictest0-21swapper/204:56:482
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional