You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-07 - 19:25

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot3s.osadl.org (updated Wed Jan 07, 2026 12:43:55)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
902486212262,56sleep30-21swapper/307:07:383
90254828458,9sleep20-21swapper/207:08:252
90246428352,10sleep10-21swapper/107:07:191
90240427557,13sleep00-21swapper/007:06:330
902801992522,1cyclictest0-21swapper/312:10:013
902801992522,1cyclictest0-21swapper/307:34:593
902801992521,2cyclictest0-21swapper/307:39:563
902801992520,2cyclictest0-21swapper/311:00:133
10276562220,9sleep134-21ksoftirqd/109:05:141
902801992118,1cyclictest0-21swapper/309:54:593
902801992118,1cyclictest0-21swapper/308:50:173
10452632210,2sleep00-21swapper/009:20:160
90280199201,14cyclictest0-21swapper/307:15:133
90280199201,11cyclictest1009057-21taskset08:45:153
90280199199,8cyclictest1195503-21turbostat11:49:553
90280199192,13cyclictest1132423-21turbostat10:45:003
902801991916,1cyclictest0-21swapper/311:40:013
902801991915,2cyclictest0-21swapper/311:55:133
902801991915,2cyclictest0-21swapper/308:05:013
90280199191,13cyclictest0-21swapper/312:00:113
902801991910,6cyclictest968413-21turbostat08:10:003
12268712192,12sleep01226892-21munin-run12:15:010
902801991816,1cyclictest0-21swapper/310:25:173
902801991816,1cyclictest0-21swapper/309:55:023
90280199181,14cyclictest0-21swapper/312:25:123
90280199181,13cyclictest984679-21chrt08:25:113
90280199181,13cyclictest11687452chrt11:16:103
90280199181,13cyclictest0-21swapper/310:50:123
90280199181,13cyclictest0-21swapper/309:40:113
90280199181,10cyclictest1032529-21taskset09:10:013
90280199180,11cyclictest947397-21taskset07:50:013
12028492181,8sleep21204614-21tr11:50:172
11009712181,13sleep01101049-21cpuspeed_turbos10:15:110
10377952181,8sleep21037937-21cut09:15:142
90280199178,6cyclictest1184392-21taskset11:30:413
902801991714,1cyclictest0-21swapper/311:25:183
902801991714,1cyclictest0-21swapper/311:10:163
902801991714,1cyclictest0-21swapper/310:15:153
902801991714,1cyclictest0-21swapper/308:55:133
902801991714,1cyclictest0-21swapper/308:15:013
902801991714,1cyclictest0-21swapper/307:10:133
902801991713,2cyclictest0-21swapper/311:40:143
90280199171,13cyclictest1205275-21taskset11:50:253
90280199171,13cyclictest0-21swapper/310:20:133
90280199171,10cyclictest920761-21taskset07:25:113
90280199171,10cyclictest1236984-21taskset12:20:283
90280199171,10cyclictest1036546-21taskset09:10:203
11593932178,4sleep21160635-21cut11:10:172
90280199161,9cyclictest953017-21taskset07:55:133
90280199161,9cyclictest1037938-21taskset09:15:143
902801991614,1cyclictest0-21swapper/308:40:263
90280199161,3cyclictest0-21swapper/310:35:113
902801991613,1cyclictest0-21swapper/312:35:163
902801991613,1cyclictest0-21swapper/310:55:193
902801991613,1cyclictest0-21swapper/310:10:163
902801991613,1cyclictest0-21swapper/310:05:183
902801991613,1cyclictest0-21swapper/309:45:253
902801991613,1cyclictest0-21swapper/309:30:173
902801991613,1cyclictest0-21swapper/309:20:213
90280199161,2cyclictest0-21swapper/310:00:103
902801991612,2cyclictest0-21swapper/309:35:273
90280199161,13cyclictest0-21swapper/307:45:103
90280199161,12cyclictest1158589-21sleep11:10:003
90280199161,12cyclictest0-21swapper/307:40:213
90280199161,10cyclictest999665-21chrt08:35:253
90280199160,13cyclictest0-21swapper/309:25:153
90280199160,13cyclictest0-21swapper/307:20:193
90280199151,9cyclictest994470-21taskset08:30:253
90280199151,9cyclictest1232091-21taskset12:19:473
90280199151,9cyclictest1226536-21taskset12:10:453
90280199151,11cyclictest1121646-21turbostat10:35:013
90280199151,10cyclictest1243620-21chrt12:30:143
90280199151,10cyclictest1169848-21chrt11:20:153
90280199150,13cyclictest0-21swapper/309:00:013
90280199150,13cyclictest0-21swapper/308:20:023
90280199150,12cyclictest0-21swapper/310:45:203
90280199150,12cyclictest0-21swapper/308:15:153
11818062151,9sleep11182858-21tr11:30:161
10164742151,9sleep01016492-21apt08:55:120
10152692151,9sleep21015325-21ntp_states08:50:202
9157502141,9sleep1915961-21sh07:20:141
11243342141,8sleep21125987-21sed10:35:172
11151022141,8sleep11111421-21/usr/sbin/munin10:25:181
10749752147,3sleep01075168-21grep09:50:150
10117232141,8sleep01011224-21/usr/sbin/munin08:50:140
9954832131,8sleep2995197-21/usr/sbin/munin08:35:132
9210192121,7sleep0921075-21fschecks_count07:25:130
9156262121,1sleep00-21swapper/007:20:130
12483672121,7sleep01248522-21cstates12:35:120
11675012121,7sleep11167895-21missed_timers11:15:181
11470002121,6sleep11147048-21ntpq10:55:221
11027952121,7sleep11104574-21tr10:15:161
10856952121,3sleep21085880-21tr10:00:152
10589432121,7sleep21058968-21/usr/sbin/munin09:35:142
10325632120,2sleep20-21swapper/209:10:122
9738982111,5sleep2973916-21apt-get08:15:132
9639012111,5sleep0964017-21irqstats08:05:140
9461982111,5sleep1946252-21ntpq07:45:181
9212862111,6sleep2921343-21kthreadcore07:25:152
12432832111,5sleep11243406-21diskstats12:30:131
11484212111,6sleep01148616-21diskstats11:00:130
11384302111,6sleep11138487-21kthreadcore10:50:151
11313152111,5sleep21131404-21ntpq10:40:192
10956932111,2sleep01095712-21apt-get10:10:110
9899382104,2sleep1990194-21df08:30:111
9834192101,5sleep0983465-21ntpq08:20:200
9581682101,4sleep1957655-21apt-get08:00:021
9526682101,4sleep0952687-21apt-get07:55:110
9480052101,5sleep2949521-21grep07:50:172
9461462101,4sleep0946196-21ntpq07:45:170
9368212101,3sleep187-21kswapd007:40:061
9368202101,4sleep0936501-31find07:40:050
9267702100,6sleep2927044-21kthreadcore07:30:172
9080002101,5sleep20-21swapper/207:15:012
12486352101,4sleep21248848-21hddtemp_smartct12:35:142
12205952101,4sleep11220636-21ntpq12:05:191
12119082101,4sleep21212024-21irqstats12:00:152
12049172101,4sleep01204947-21ntpq11:50:200
11770972101,4sleep01178340-21memory11:25:190
11732992101,5sleep01173350-21ntpq11:20:210
11574632101,4sleep11157552-21ntpq11:05:201
11488272101,4sleep21148897-21irqstats11:00:152
11369682101,5sleep11137207-21grep10:45:281
11274862101,4sleep01127724-21df_inode10:40:120
11206232101,4sleep21120670-21ntpq10:30:212
10816942100,6sleep21083490-21cut09:55:172
10804292101,5sleep01080500-21/usr/sbin/munin09:55:150
10691852101,4sleep01069220-21apt-get09:45:090
10470652101,5sleep11047191-21run-parts09:20:211
10328432101,5sleep01032917-21sort09:10:140
10263732101,5sleep11026568-21/usr/sbin/munin09:00:251
10165592101,4sleep21016762-21diskstats08:55:142
999216291,3sleep1999263-21ntpq08:35:211
993920291,3sleep0993966-21ntpq08:30:180
985174291,4sleep2985411-21cut08:25:152
983704291,4sleep1983877-21sort08:20:251
968684291,4sleep2968937-21df08:10:132
956726291,4sleep2956774-21/usr/sbin/munin07:55:212
953819291,4sleep1955521-21cut07:55:161
936466291,4sleep0936501-31find07:39:560
932395293,3sleep20-21swapper/207:35:172
903270290,1sleep10-21swapper/107:10:161
1243086290,1sleep00-21swapper/012:30:040
1237627291,4sleep21237781-21cat12:25:122
1236756291,4sleep11236973-21users12:20:271
1225924291,3sleep11225978-21ntpq12:10:211
1213101291,4sleep01214855-21cut12:00:180
1194514291,3sleep11194561-21ntpq11:40:181
1159321290,1sleep10-21swapper/111:10:161
1132745291,4sleep01132961-21/usr/sbin/munin10:45:140
1122771291,4sleep11124333-21grep10:35:161
1121628291,4sleep11121772-21apt-config10:35:011
1097501291,5sleep20-21swapper/210:10:172
1096265291,4sleep11097499-21cut10:10:161
1090438291,3sleep00-21swapper/010:05:130
1069511291,4sleep21069614-21perl09:45:112
1064253291,4sleep11064321-21fschecks_time09:40:141
1058668291,2sleep00-21swapper/009:35:110
1052485291,4sleep11052745-21sed09:25:261
1052213291,4sleep01052263-21ld09:25:210
1027603291,4sleep01027654-21sort09:05:130
1009611291,4sleep01009684-21/usr/sbin/munin08:45:180
974180281,3sleep0974304-21grep08:15:160
969196281,3sleep1969515-21kthreadcore08:10:151
968399281,3sleep2968630-21latency_hist08:10:002
958489281,3sleep2958203-21/usr/sbin/munin08:00:152
947413280,3sleep1947432-21apt-get07:50:121
942457281,4sleep20-21swapper/207:45:112
931254280,1sleep10-21swapper/107:35:001
926229280,2sleep00-21swapper/007:30:030
907942280,1sleep00-21swapper/007:14:560
1252398281,3sleep11252455-21ntp_states12:35:211
1247197280,1sleep20-21swapper/212:30:222
1238110281,3sleep11238198-21kernelversion12:25:151
1231415281,3sleep11231550-21/usr/sbin/munin12:15:251
1221655280,1sleep20-21swapper/212:10:022
1199637281,2sleep11199735-21ntpq11:45:191
1189874280,1sleep20-21swapper/211:39:112
1153571281,3sleep21153587-21/usr/sbin/munin11:05:132
1143645281,3sleep21144953-21kthreadcore10:55:182
1137404280,1sleep20-21swapper/210:47:162
1068231280,1sleep20-21swapper/209:40:262
1058403281,3sleep0553-21dbus-daemon09:35:000
1043516281,3sleep21043613-21ps09:20:142
1027871281,3sleep20-21swapper/209:05:162
1015896280,1sleep10-21swapper/108:51:041
1007295281,3sleep21007768-21kthreadcore08:45:162
1001051281,3sleep00-21swapper/008:40:160
989581270,1sleep00-21swapper/008:29:070
983526270,4sleep20-21swapper/208:20:232
964018270,3sleep1964053-21kthreadcore08:05:141
958772271,2sleep00-21swapper/008:00:160
936822270,2sleep20-21swapper/207:40:072
925508270,1sleep10-21swapper/107:27:271
915118270,5sleep00-21swapper/007:19:160
903483273,1sleep2431ktimers/207:10:162
1210864270,1sleep10-21swapper/111:56:091
1201034271,2sleep10-21swapper/111:50:131
1196316271,3sleep20-21swapper/211:45:172
1190501270,2sleep00-21swapper/011:40:100
1184989270,2sleep10-21swapper/111:35:101
1175083271,2sleep20-21swapper/211:25:172
1142764270,1sleep00-21swapper/010:54:270
1116096270,1sleep20-21swapper/210:26:162
1113320273,1sleep00-21swapper/010:25:170
1106780271,3sleep00-21swapper/010:20:160
1083895271,3sleep11083943-21ntpq09:55:201
1074487271,3sleep11074709-21cstates09:50:121
1063348270,1sleep10-21swapper/109:35:441
1052041271,2sleep21052154-21ntpq09:25:202
1037773270,2sleep10-21swapper/109:15:131
1037322270,1sleep10-21swapper/109:12:401
1021281270,1sleep10-21swapper/108:57:221
1006303271,2sleep10-21swapper/108:45:151
1002165271,3sleep10-21swapper/108:40:171
995766261,2sleep00-21swapper/008:35:160
994626260,3sleep2421rcuc/208:31:092
989640260,1sleep10-21swapper/108:29:521
978572260,1sleep10-21swapper/108:16:171
973313260,1sleep00-21swapper/008:10:590
951725261,2sleep00-21swapper/007:50:260
915459260,2sleep20-21swapper/207:20:102
910199261,2sleep10-21swapper/107:15:121
1241382260,2sleep01241527-21strings12:25:180
1232394260,1sleep20-21swapper/212:20:122
1231955260,1sleep20-21swapper/212:18:062
1227156260,2sleep00-21swapper/012:15:120
1221295260,1sleep00-21swapper/012:05:580
1210244260,3sleep21210289-21open_files11:55:232
1206223260,2sleep00-21swapper/011:55:120
1193024260,2sleep20-21swapper/211:40:152
1179490260,1sleep10-21swapper/111:29:571
1174049260,1sleep10-21swapper/111:22:021
1165758261,2sleep00-21swapper/011:15:170
1151979260,2sleep11152248-21strings11:00:181
1146736260,2sleep01146994-21strings10:55:210
1138868261,2sleep20-21swapper/210:50:162
1127725261,2sleep11127855-21gltestperf10:40:131
1126914260,1sleep00-21swapper/010:37:010
1119748261,2sleep00-21swapper/010:30:180
1110889260,1sleep10-21swapper/110:21:391
1109635260,2sleep21110108-21strings10:20:192
1095388260,1sleep20-21swapper/210:09:282
1091029261,2sleep10-21swapper/110:05:181
1089646261,2sleep10-21swapper/110:00:261
1070454261,2sleep10-21swapper/109:45:141
1041499260,2sleep01041672-21strings09:15:190
1023745261,2sleep00-21swapper/009:00:160
1004786261,2sleep20-21swapper/208:40:262
1216665251,2sleep20-21swapper/212:05:122
1211451250,1sleep10-21swapper/112:00:111
1196038250,2sleep00-21swapper/011:45:140
1183728250,1sleep00-21swapper/011:30:190
1164506250,1sleep20-21swapper/211:15:152
1158433250,4sleep00-21swapper/011:08:270
1104575250,1sleep20-21swapper/210:15:172
1089884250,3sleep00-21swapper/010:01:220
1079058252,2sleep2902796-21cyclictest09:50:482
1022504250,1sleep20-21swapper/209:00:142
926238240,1sleep10-21swapper/107:30:041
1232110240,1sleep00-21swapper/012:20:020
1185283240,1sleep00-21swapper/011:35:120
1179498240,1sleep20-21swapper/211:30:012
1169335240,1sleep20-21swapper/211:20:112
1158817240,1sleep00-21swapper/011:10:110
1063750240,1sleep00-21swapper/009:40:010
1053598240,1sleep20-21swapper/209:30:122
1053454240,1sleep10-21swapper/109:30:111
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional