You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-16 - 00:46

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot6s.osadl.org (updated Tue Jul 15, 2025 12:43:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3027599944,82cyclictest0-21swapper/008:58:570
3028199935,26cyclictest0-21swapper/211:03:222
3028199933,28cyclictest0-21swapper/207:49:152
3028199925,27cyclictest0-21swapper/211:49:002
3028199924,27cyclictest0-21swapper/209:39:112
3028199924,27cyclictest0-21swapper/209:19:092
3028199924,25cyclictest0-21swapper/206:54:012
3027599924,34cyclictest0-21swapper/010:46:220
3028199913,27cyclictest0-21swapper/209:53:432
3028199913,27cyclictest0-21swapper/209:53:432
30275999025,7cyclictest1265-21runrttasks08:18:310
3028199884,28cyclictest0-21swapper/210:04:172
30281998818,14cyclictest0-21swapper/210:13:132
3027599884,76cyclictest0-21swapper/007:39:140
30275998825,56cyclictest0-21swapper/006:44:200
3028199874,27cyclictest0-21swapper/211:47:552
30281998731,49cyclictest23219-21needreboot09:29:122
3027599874,74cyclictest0-21swapper/006:24:180
3027599874,25cyclictest0-21swapper/008:33:570
3027599864,27cyclictest0-21swapper/007:48:590
3027599864,13cyclictest0-21swapper/009:58:550
3028199855,29cyclictest0-21swapper/208:33:592
3027599853,32cyclictest19451-21crond07:08:480
3027599853,32cyclictest19451-21crond07:08:480
3028199846,22cyclictest0-21swapper/207:18:492
3028199844,28cyclictest0-21swapper/209:54:012
3028199844,28cyclictest0-21swapper/209:54:012
3027599844,72cyclictest0-21swapper/006:49:170
3028199835,28cyclictest0-21swapper/206:34:072
3027599834,72cyclictest0-21swapper/010:23:410
3027599833,25cyclictest13254-21grep09:09:040
3027599833,25cyclictest13254-21grep09:09:040
3028199825,27cyclictest0-21swapper/207:57:512
3027599825,32cyclictest31712-21processes10:54:090
3027599824,33cyclictest0-21swapper/009:33:560
3027599824,13cyclictest0-21swapper/007:54:550
3027599823,72cyclictest0-21swapper/007:36:570
3027599823,72cyclictest0-21swapper/007:14:030
3027599823,33cyclictest2170-21sshd09:54:000
3027599823,33cyclictest2170-21sshd09:54:000
30275998214,8cyclictest0-21swapper/011:03:390
3028199815,27cyclictest0-21swapper/207:29:082
3028199815,25cyclictest0-21swapper/208:57:582
3028199814,23cyclictest0-21swapper/206:44:032
3028199814,13cyclictest0-21swapper/208:14:062
3028199814,13cyclictest0-21swapper/207:12:432
3028199814,11cyclictest0-21swapper/209:38:432
3028199813,13cyclictest0-21swapper/208:08:592
30281998111,20cyclictest0-21swapper/210:33:402
3027599814,69cyclictest0-21swapper/009:44:080
3027599814,22cyclictest0-21swapper/011:34:120
3027599813,29cyclictest1265-21runrttasks08:28:580
30275998123,7cyclictest1265-21runrttasks11:22:190
3028199805,11cyclictest0-21swapper/208:29:092
3028199804,17cyclictest0-21swapper/209:58:562
3028199803,32cyclictest0-21swapper/208:59:202
30281998020,9cyclictest2839-21aten2.4-expect11:03:572
30281998010,18cyclictest0-21swapper/208:19:012
3027599805,11cyclictest0-21swapper/010:09:120
3027599803,27cyclictest0-21swapper/008:49:210
30275998026,8cyclictest4198-21uname06:34:160
3028199795,65cyclictest0-21swapper/208:28:452
3028199795,65cyclictest0-21swapper/208:28:452
3028199795,25cyclictest0-21swapper/211:23:592
3028199795,23cyclictest0-21swapper/211:23:172
3028199794,34cyclictest0-21swapper/207:39:182
3028199794,12cyclictest0-21swapper/210:58:212
3028199793,27cyclictest0-21swapper/211:28:582
3028199793,24cyclictest31837-21proc_pri06:24:202
30281997925,8cyclictest18503-21needreboot07:04:142
30281997925,8cyclictest18503-21needreboot07:04:142
30281997918,12cyclictest0-21swapper/210:14:102
30281997912,16cyclictest0-21swapper/207:23:472
3027599793,25cyclictest0-21swapper/009:04:150
30275997925,8cyclictest1265-21runrttasks08:48:340
30275997923,8cyclictest1265-21runrttasks08:00:010
3028199789,11cyclictest0-21swapper/208:07:532
3028199785,15cyclictest0-21swapper/210:18:532
3028199784,32cyclictest0-21swapper/206:39:082
3028199784,22cyclictest1265-21runrttasks09:15:452
3028199783,27cyclictest11954-21tune2fs07:59:062
30281997820,10cyclictest0-21swapper/206:49:122
30281997818,13cyclictest0-21swapper/207:47:492
30281997818,13cyclictest0-21swapper/207:47:492
3027599784,15cyclictest0-21swapper/011:28:530
3027599783,69cyclictest0-21swapper/008:39:210
3027599783,68cyclictest0-21swapper/008:09:090
3027599783,29cyclictest19396-21munin-plugin-st09:23:430
30275997822,9cyclictest16644-21grep10:24:000
30275997819,14cyclictest0-21swapper/010:28:590
3028199775,14cyclictest0-21swapper/210:48:522
3028199774,30cyclictest0-21swapper/206:59:032
3028199774,26cyclictest0-21swapper/207:37:482
3028199774,23cyclictest0-21swapper/209:03:552
30281997720,8cyclictest0-21swapper/210:33:582
30281997720,8cyclictest0-21swapper/210:33:582
30281997712,13cyclictest0-21swapper/207:24:192
3027599774,31cyclictest0-21swapper/007:03:200
3027599774,26cyclictest15684-21sed09:14:050
3027599774,25cyclictest0-21swapper/006:54:100
3027599774,24cyclictest0-21swapper/011:49:100
30275997728,8cyclictest14879-21needreboot08:04:120
30275997725,10cyclictest1265-21runrttasks11:18:360
30275997725,10cyclictest1265-21runrttasks11:18:360
30275997723,8cyclictest6461-21aten2.4-expect10:03:550
3028199765,32cyclictest0-21swapper/211:34:062
3028199765,14cyclictest0-21swapper/210:44:022
3028199764,31cyclictest0-21swapper/208:39:132
3028199764,30cyclictest0-21swapper/209:44:072
3028199764,29cyclictest0-21swapper/208:52:142
30281997620,13cyclictest0-21swapper/209:24:172
3027599764,25cyclictest0-21swapper/007:13:490
3027599763,65cyclictest0-21swapper/010:43:400
3027599763,31cyclictest0-21swapper/007:33:470
30275997619,9cyclictest1265-21runrttasks06:29:150
3028199754,25cyclictest0-21swapper/211:39:032
3028199753,30cyclictest0-21swapper/210:43:412
3028199753,24cyclictest0-21swapper/208:43:562
3027599754,30cyclictest0-21swapper/009:49:550
3027599754,30cyclictest0-21swapper/009:49:550
30275997523,8cyclictest1265-21runrttasks10:51:190
3028199744,24cyclictest0-21swapper/209:08:592
3028199744,24cyclictest0-21swapper/209:08:592
3028199743,11cyclictest11377-1kworker/2:2H10:28:402
30281997413,14cyclictest0-21swapper/211:14:122
30281997413,14cyclictest0-21swapper/211:14:122
3027599744,30cyclictest0-21swapper/008:54:080
3027599743,26cyclictest0-21swapper/011:04:040
3027599743,23cyclictest0-21swapper/008:19:160
3027599743,13cyclictest0-21swapper/007:28:340
30275997417,7cyclictest13386-21latency_hist10:18:410
189402740,3sleep3361ktimersoftd/311:34:133
3027599733,31cyclictest0-21swapper/007:44:030
3027599733,31cyclictest0-21swapper/007:44:030
30275997329,8cyclictest22653-21tune2fs09:29:050
30275997325,8cyclictest20227-21fschecks_time11:38:570
3027599724,26cyclictest0-21swapper/007:19:250
3027599723,11cyclictest0-21swapper/011:44:130
30275997223,7cyclictest23632-21cstates08:24:000
30275997223,7cyclictest23632-21cstates08:24:000
30275997223,7cyclictest1265-21runrttasks10:37:450
30275997223,7cyclictest1265-21runrttasks10:37:450
30275997120,14cyclictest0-21swapper/006:39:050
30275997118,6cyclictest28933-21latency_hist09:43:430
2939427111,48sleep00-21swapper/006:19:160
2939427111,48sleep00-21swapper/006:19:160
3027599694,26cyclictest13207-21gltestperf11:24:020
3028199674,15cyclictest0-21swapper/211:08:532
3028199673,15cyclictest0-21swapper/206:29:052
223099610,5rtkit-daemon2229-21rtkit-daemon08:09:051
30282995940,11cyclictest16916-21crond11:33:383
223099590,4rtkit-daemon2229-21rtkit-daemon10:01:081
223099590,18rtkit-daemon2229-21rtkit-daemon07:25:221
31882583,11sleep10-21swapper/111:04:011
30280995824,14cyclictest0-21swapper/111:39:011
3027599583,49cyclictest0-21swapper/011:09:070
223099580,4rtkit-daemon2229-21rtkit-daemon11:02:431
223099580,19rtkit-daemon2229-21rtkit-daemon10:10:151
30282995715,21cyclictest0-21swapper/309:48:563
30282995715,21cyclictest0-21swapper/309:48:563
3028099574,29cyclictest0-21swapper/107:48:471
3028099574,29cyclictest0-21swapper/107:48:471
30280995718,13cyclictest0-21swapper/106:34:131
3028299564,13cyclictest0-21swapper/308:53:583
30282995622,14cyclictest0-21swapper/310:18:413
30280995623,9cyclictest31762-21ps08:39:151
30280995620,13cyclictest0-21swapper/110:54:081
30280995618,13cyclictest0-21swapper/110:24:111
3028299555,29cyclictest0-21swapper/311:48:563
3028299555,29cyclictest0-21swapper/306:44:013
30282995536,10cyclictest16502-21df_abs08:09:033
30282995522,14cyclictest0-21swapper/310:53:543
30282995522,14cyclictest0-21swapper/308:18:473
3028099556,28cyclictest0-21swapper/109:48:571
3028099556,28cyclictest0-21swapper/109:48:571
3028099555,30cyclictest0-21swapper/110:18:421
3028099555,27cyclictest0-21swapper/106:44:041
3028099554,27cyclictest0-21swapper/107:54:181
30280995521,14cyclictest0-21swapper/109:39:151
30280995521,13cyclictest0-21swapper/107:14:091
30280995521,12cyclictest0-21swapper/109:18:451
30280995520,12cyclictest0-21swapper/110:18:551
223099550,4rtkit-daemon2229-21rtkit-daemon07:40:211
170232553,44sleep317030-21iostat_ios10:24:053
37292543,11sleep30-21swapper/307:41:453
3028299545,29cyclictest0-21swapper/311:09:073
3028299545,29cyclictest0-21swapper/310:08:563
3028299545,29cyclictest0-21swapper/309:24:113
3028299544,30cyclictest0-21swapper/310:59:103
3028299544,28cyclictest0-21swapper/307:14:013
3028299544,11cyclictest0-21swapper/308:26:263
3028299544,11cyclictest0-21swapper/308:26:263
30282995433,13cyclictest8368-21cut08:59:043
3028299543,33cyclictest0-21swapper/309:18:263
3028099545,29cyclictest0-21swapper/108:24:141
3028099545,29cyclictest0-21swapper/108:24:141
3028099545,28cyclictest0-21swapper/108:19:161
3028099545,27cyclictest0-21swapper/107:33:481
3028099544,30cyclictest0-21swapper/110:38:591
3028099544,28cyclictest0-21swapper/107:53:481
3028099544,27cyclictest0-21swapper/110:08:421
30280995421,13cyclictest0-21swapper/106:39:111
30280995416,19cyclictest0-21swapper/108:01:141
204992544,44sleep10-21swapper/110:33:131
3028299537,27cyclictest0-21swapper/308:38:593
3028299535,29cyclictest0-21swapper/309:04:113
3028299535,29cyclictest0-21swapper/307:59:003
3028299534,30cyclictest0-21swapper/308:44:083
3028299534,29cyclictest0-21swapper/311:24:143
3028299534,29cyclictest0-21swapper/310:34:163
3028299534,29cyclictest0-21swapper/310:34:163
3028299534,29cyclictest0-21swapper/308:04:043
3028299534,28cyclictest0-21swapper/311:14:113
3028299534,28cyclictest0-21swapper/311:14:113
30282995336,10cyclictest7313-21gltestperf10:04:033
30282995320,13cyclictest0-21swapper/306:33:513
3028099535,28cyclictest0-21swapper/109:24:171
3028099534,27cyclictest0-21swapper/108:34:171
30280995337,8cyclictest0-21swapper/109:44:041
30280995321,13cyclictest0-21swapper/110:34:031
30280995321,13cyclictest0-21swapper/110:34:031
223099530,5rtkit-daemon2229-21rtkit-daemon07:34:061
3028299525,29cyclictest0-21swapper/310:43:543
3028299525,29cyclictest0-21swapper/307:54:193
3028299524,29cyclictest0-21swapper/311:08:233
3028299524,29cyclictest0-21swapper/306:34:263
30282995230,13cyclictest25298-21head07:19:123
3028299523,25cyclictest0-21swapper/306:59:033
30282995220,14cyclictest0-21swapper/308:29:183
30282995220,13cyclictest0-21swapper/309:43:433
3028099525,30cyclictest0-21swapper/111:43:581
3028099525,28cyclictest0-21swapper/111:14:021
3028099525,28cyclictest0-21swapper/111:14:021
3028099525,27cyclictest0-21swapper/111:28:141
3028099525,23cyclictest25752-21python11:49:081
3028099524,40cyclictest0-21swapper/109:18:471
3028099524,27cyclictest0-21swapper/107:04:161
3028099524,27cyclictest0-21swapper/107:04:161
3028099524,25cyclictest0-21swapper/109:03:021
30280995233,12cyclictest0-21swapper/106:24:221
30280995232,12cyclictest0-21swapper/108:28:581
30280995231,14cyclictest0-21swapper/107:19:241
30280995221,10cyclictest0-21swapper/110:49:111
3027599524,42cyclictest0-21swapper/009:24:180
223099520,5rtkit-daemon2229-21rtkit-daemon10:43:591
203062523,42sleep10-21swapper/108:14:481
114652523,10sleep10-21swapper/106:49:191
3028299515,28cyclictest0-21swapper/311:19:123
3028299515,28cyclictest0-21swapper/310:29:053
3028299515,28cyclictest0-21swapper/309:09:053
3028299515,28cyclictest0-21swapper/309:09:053
3028299515,27cyclictest0-21swapper/310:38:523
3028299515,27cyclictest0-21swapper/309:44:183
3028299514,28cyclictest0-21swapper/307:24:383
3028299514,28cyclictest0-21swapper/307:04:183
3028299514,28cyclictest0-21swapper/307:04:183
3028299514,28cyclictest0-21swapper/306:53:493
3028099515,27cyclictest0-21swapper/109:13:041
3028099515,27cyclictest0-21swapper/109:13:041
3028099514,27cyclictest0-21swapper/108:54:181
30280995130,14cyclictest0-21swapper/108:49:011
239712513,41sleep10-21swapper/109:31:181
3028299505,28cyclictest0-21swapper/310:53:203
3028299505,27cyclictest0-21swapper/311:43:533
3028299504,28cyclictest0-21swapper/309:18:583
3028299504,26cyclictest6693-21sshd06:39:203
30282995032,11cyclictest2517-21cut09:54:043
30282995032,11cyclictest2517-21cut09:54:043
30282995020,11cyclictest1265-21runrttasks08:19:183
30282995019,13cyclictest0-21swapper/307:29:233
30280995031,12cyclictest0-21swapper/108:44:081
3028099503,42cyclictest0-21swapper/107:09:181
110402503,40sleep10-21swapper/109:04:061
3028299494,27cyclictest0-21swapper/308:37:583
3028299494,13cyclictest0-21swapper/307:09:043
30282994918,14cyclictest0-21swapper/310:20:173
3028099494,27cyclictest18859-21grep11:34:121
30280994932,10cyclictest15231-21cpuspeed_turbos06:59:051
241052493,40sleep30-21swapper/309:33:063
223099490,4rtkit-daemon2229-21rtkit-daemon08:04:201
3028299484,29cyclictest0-21swapper/307:34:393
30282994818,12cyclictest0-21swapper/306:24:283
3028099484,27cyclictest0-21swapper/109:54:151
3028099484,27cyclictest0-21swapper/109:54:151
30280994830,11cyclictest0-21swapper/111:19:051
30280994829,12cyclictest0-21swapper/111:33:381
27422482,11sleep10-21swapper/106:33:511
263972483,13sleep10-21swapper/109:36:511
59612473,10sleep30-21swapper/307:44:393
59612473,10sleep30-21swapper/307:44:393
3028299474,26cyclictest0-21swapper/309:59:203
30282994733,8cyclictest25215-21gltestperf09:34:033
3028299473,26cyclictest0-21swapper/308:49:113
223099470,4rtkit-daemon2229-21rtkit-daemon11:08:581
214642473,11sleep30-21swapper/311:40:473
30282994528,11cyclictest13389-21gltestperf06:54:083
3028299447,23cyclictest0-21swapper/306:23:513
3028299447,23cyclictest0-21swapper/306:23:513
301492444,33sleep10-21swapper/106:22:391
301492444,33sleep10-21swapper/106:22:391
3028299433,13cyclictest7397-21grep07:49:063
30280994319,8cyclictest12531-21munin-node06:54:071
292962415,28sleep20-21swapper/206:19:142
292962415,28sleep20-21swapper/206:19:142
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional