diff --git a/deploy/00crds.yaml b/deploy/00crds.yaml index e9963a2..8daaaac 100644 --- a/deploy/00crds.yaml +++ b/deploy/00crds.yaml @@ -878,6 +878,356 @@ spec: --- apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.17.1 + name: migrationblueprints.vjailbreak.k8s.pf9.io +spec: + group: vjailbreak.k8s.pf9.io + names: + kind: MigrationBlueprint + listKind: MigrationBlueprintList + plural: migrationblueprints + singular: migrationblueprint + scope: Namespaced + versions: + - additionalPrinterColumns: + - jsonPath: .spec.displayName + name: Display Name + type: string + - jsonPath: .spec.vmwareRef + name: Source + type: string + - jsonPath: .spec.pcdRef + name: Destination + type: string + - jsonPath: .metadata.creationTimestamp + name: Age + type: date + name: v1alpha1 + schema: + openAPIV3Schema: + description: |- + MigrationBlueprint is the Schema for the migrationblueprints API. It stores a + reusable, named migration configuration that the UI presents as a "Migration + Template": users save a configuration once and apply it later to pre-populate + the migration form. Blueprints are not read by the migration controller. + properties: + apiVersion: + description: |- + APIVersion defines the versioned schema of this representation of an object. + Servers should convert recognized schemas to the latest internal value, and + may reject unrecognized values. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources + type: string + kind: + description: |- + Kind is a string value representing the REST resource this object represents. + Servers may infer this from the endpoint the client submits requests to. + Cannot be updated. + In CamelCase. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds + type: string + metadata: + type: object + spec: + description: |- + MigrationBlueprintSpec defines a reusable, named snapshot of a migration + configuration. It captures everything from the migration form except the VM + selection, so the UI can present it as a "Migration Template" and pre-fill + the migration wizard. Blueprints are consumed only by the UI: applying one + still creates the regular MigrationTemplate/MigrationPlan/mapping resources, + so the migration controller never reads a blueprint. + properties: + advancedOptions: + description: AdvancedOptions captures the advanced migration options + to pre-fill + properties: + acknowledgeNetworkConflictRisk: + description: AcknowledgeNetworkConflictRisk indicates that the + user acknowledges the risk of network conflicts when doing live + migration + type: boolean + granularNetworks: + description: GranularNetworks is a list of networks to be migrated + items: + type: string + type: array + granularPorts: + description: GranularPorts is a list of ports to be migrated + items: + type: string + type: array + granularVolumeTypes: + description: GranularVolumeTypes is a list of volume types to + be migrated + items: + type: string + type: array + imageProfiles: + description: ImageProfiles is the ordered list of VolumeImageProfile + names to apply to the migrated VM's boot volume. + items: + type: string + type: array + networkPersistence: + description: NetworkPersistence instructs the migration helper + to persist the source networking configuration + type: boolean + periodicSyncEnabled: + description: PeriodicSyncEnabled is a boolean to enable periodic + sync + type: boolean + periodicSyncInterval: + description: PeriodicSyncInterval is the interval at which the + migration plan should be synced + type: string + removeVMwareTools: + default: true + description: |- + RemoveVMwareTools instructs the migration helper to remove VMware Tools post migration. + Defaults to true since most migrations require VMware Tools removal. + type: boolean + type: object + arrayCredsMappings: + description: |- + ArrayCredsMappings is an inline snapshot of datastore-to-ArrayCreds pairs. + Used when StorageCopyMethod is "StorageAcceleratedCopy". + items: + description: DatastoreArrayCredsMapping represents a mapping between + a VMware datastore and ArrayCreds + properties: + source: + description: Source is the name of the source datastore in VMware + type: string + target: + description: Target is the name of the ArrayCreds resource to + use for this datastore + type: string + required: + - source + - target + type: object + type: array + customMetadata: + additionalProperties: + type: string + description: |- + CustomMetadata is a map of additional key-value pairs applied as instance + metadata to every migrated VM + type: object + dataOnly: + description: |- + DataOnly copies and converts disks to Cinder volumes without creating an + OpenStack VM + type: boolean + description: + description: Description is a free-text description of the blueprint + type: string + displayName: + description: DisplayName is the human-readable name of the blueprint + shown in the UI + minLength: 1 + type: string + fallbackToDHCP: + description: FallbackToDHCP falls back to DHCP when static IP assignment + is not possible + type: boolean + firstBootScript: + description: FirstBootScript is the script to run on first boot of + migrated VMs + type: string + migrationStrategy: + description: |- + MigrationStrategy captures the migration type, cutover windows, and + health-check settings to pre-fill into the migration form. + Optional so partially filled forms can be saved as templates. + properties: + adminInitiatedCutOver: + default: false + type: boolean + arrayOffload: + default: false + type: boolean + dataCopyStart: + format: date-time + type: string + dataOnly: + default: false + description: |- + DataOnly skips OpenStack VM creation; converted Cinder volumes are left staged. + Compatible with all strategy types (hot, cold, mock). + type: boolean + disconnectSourceNetwork: + default: false + type: boolean + healthCheckPort: + default: "443" + type: string + performHealthChecks: + default: false + type: boolean + type: + enum: + - hot + - cold + - mock + type: string + vmCutoverEnd: + format: date-time + type: string + vmCutoverStart: + format: date-time + type: string + required: + - type + type: object + networkMappings: + description: |- + NetworkMappings is an inline snapshot of source-to-target network pairs. + Copied by value rather than referencing a NetworkMapping resource, since + those are per-migration objects that may be mutated or deleted. + items: + description: Network represents a mapping between source and target + networks + properties: + source: + description: Source is the name of the source network in VMware + type: string + target: + description: Target is the name of the target network in OpenStack + type: string + required: + - source + - target + type: object + type: array + noVMwareClusterFilter: + description: |- + NoVMwareClusterFilter records that the user explicitly chose the + "No Cluster" source filter: VMs on standalone ESXi hosts that are not + part of any vCenter cluster in the datacenter referenced by VMwareRef. + This is distinct from VMwareClusterName simply not being set yet, so it + is tracked as its own field rather than inferred from an empty string. + type: boolean + osFamily: + description: OSFamily is the OS type of the virtual machines this + blueprint targets + enum: + - windowsGuest + - linuxGuest + type: string + pcdRef: + description: |- + PCDRef is the name of the OpenstackCreds to use as the destination PCD environment. + Optional so partially filled forms can be saved as templates. + type: string + postMigrationAction: + description: PostMigrationAction captures the post-migration actions + to pre-fill + properties: + folderName: + type: string + moveToFolder: + type: boolean + renameVm: + type: boolean + suffix: + type: string + type: object + preserveSourceTags: + description: |- + PreserveSourceTags copies each source VM's vSphere tags and custom + attributes to the migrated VM as instance metadata + type: boolean + proxyVMRef: + description: |- + ProxyVMRef references the ProxyVM to use for data copy. + Used when StorageCopyMethod is "HotAdd". + properties: + name: + default: "" + description: |- + Name of the referent. + This field is effectively required, but due to backwards compatibility is + allowed to be empty. Instances of this type with an empty value here are + almost certainly wrong. + More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + type: string + type: object + x-kubernetes-map-type: atomic + securityGroups: + description: SecurityGroups is the list of OpenStack security group + names to apply + items: + type: string + type: array + serverGroup: + description: ServerGroup is the OpenStack server group to place migrated + VMs into + type: string + storageCopyMethod: + default: normal + description: StorageCopyMethod indicates the method to use for storage + migration + enum: + - normal + - StorageAcceleratedCopy + - HotAdd + type: string + storageMappings: + description: |- + StorageMappings is an inline snapshot of source-to-target storage pairs. + Copied by value for the same reason as NetworkMappings. + items: + description: Storage represents a mapping between source and target + storage types + properties: + source: + description: Source is the name of the source storage type in + VMware + type: string + target: + description: Target is the name of the target storage type in + OpenStack + type: string + required: + - source + - target + type: object + type: array + targetPCDClusterName: + description: TargetPCDClusterName is the name of the PCD cluster to + migrate VMs into + type: string + useGPUFlavor: + description: UseGPUFlavor indicates if the migration should filter + and use GPU-enabled flavors + type: boolean + virtioWinDriver: + description: VirtioWinDriver is the virtio-win driver version to use + for Windows guests + type: string + vmwareClusterName: + description: |- + VMwareClusterName is the name of the source vCenter cluster VMs are + migrated from. + type: string + vmwareRef: + description: |- + VMwareRef is the name of the VMwareCreds to use as the source environment. + Optional so partially filled forms can be saved as templates. + type: string + required: + - displayName + type: object + type: object + served: true + storage: true + subresources: {} +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition metadata: annotations: controller-gen.kubebuilder.io/version: v0.17.1 @@ -976,10 +1326,20 @@ spec: migration plan should be synced type: string removeVMwareTools: - description: RemoveVMwareTools instructs the migration helper - to remove VMware Tools post migration + default: true + description: |- + RemoveVMwareTools instructs the migration helper to remove VMware Tools post migration. + Defaults to true since most migrations require VMware Tools removal. type: boolean type: object + customMetadata: + additionalProperties: + type: string + description: |- + CustomMetadata is a map of additional key-value pairs applied as instance + metadata to every migrated VM in the plan. Keys here override colliding + keys derived from preserved source tags. + type: object fallbackToDHCP: type: boolean firstBootScript: @@ -998,6 +1358,12 @@ spec: dataCopyStart: format: date-time type: string + dataOnly: + default: false + description: |- + DataOnly skips OpenStack VM creation; converted Cinder volumes are left staged. + Compatible with all strategy types (hot, cold, mock). + type: boolean disconnectSourceNetwork: default: false type: boolean @@ -1069,6 +1435,12 @@ spec: suffix: type: string type: object + preserveSourceTags: + default: false + description: |- + PreserveSourceTags copies each source VM's vSphere tags and custom attributes + to the migrated VM as instance metadata. Applies to all VMs in the plan. + type: boolean securityGroups: items: type: string @@ -1169,6 +1541,10 @@ spec: spec: description: Spec defines the desired state of Migration properties: + dataOnly: + description: DataOnly indicates no OpenStack VM should be created + after disk conversion. + type: boolean disconnectSourceNetwork: description: |- DisconnectSourceNetwork specifies whether to disconnect the source VM's network interfaces @@ -1202,6 +1578,12 @@ spec: podRef: description: PodRef is the name of the pod type: string + preserveSourceTags: + default: false + description: |- + PreserveSourceTags indicates whether the source VM's vSphere tags and custom + attributes are copied to the migrated VM as instance metadata. + type: boolean vmName: description: VMName is the name of the VM getting migrated from VMWare to Openstack @@ -1297,6 +1679,7 @@ spec: - IdentifyingBlockDevices - HotAddTransferInProgress - HotAddCleanup + - DataCopied type: string retryable: description: |- @@ -1304,6 +1687,12 @@ spec: Set to false for VMs with RDM (Raw Device Mapping) disks that share storage, as RDM disk migration state prevents automatic retry. type: boolean + stagedVolumeIDs: + description: StagedVolumeIDs lists the Cinder volume IDs created during + a data-only migration. + items: + type: string + type: array syncWarningMessage: description: |- SyncWarningMessage contains the warning message when periodic sync is in a warning state. @@ -1440,10 +1829,6 @@ spec: description: TargetPCDClusterName is the name of the PCD cluster where the virtual machine will be migrated type: string - useFlavorless: - description: UseFlavorless indicates if the migration should use flavorless - VM creation for PCD. - type: boolean useGPUFlavor: description: UseGPUFlavor indicates if the migration should filter and use GPU-enabled flavors. @@ -2410,8 +2795,10 @@ spec: migration plan should be synced type: string removeVMwareTools: - description: RemoveVMwareTools instructs the migration helper - to remove VMware Tools post migration + default: true + description: |- + RemoveVMwareTools instructs the migration helper to remove VMware Tools post migration. + Defaults to true since most migrations require VMware Tools removal. type: boolean type: object bmConfigRef: @@ -2509,6 +2896,14 @@ spec: - vmSequence type: object type: array + customMetadata: + additionalProperties: + type: string + description: |- + CustomMetadata is a map of additional key-value pairs applied as instance + metadata to every migrated VM in the plan. Keys here override colliding + keys derived from preserved source tags. + type: object firstBootScript: default: echo "Add your startup script here!" type: string @@ -2525,6 +2920,12 @@ spec: dataCopyStart: format: date-time type: string + dataOnly: + default: false + description: |- + DataOnly skips OpenStack VM creation; converted Cinder volumes are left staged. + Compatible with all strategy types (hot, cold, mock). + type: boolean disconnectSourceNetwork: default: false type: boolean @@ -2566,6 +2967,12 @@ spec: suffix: type: string type: object + preserveSourceTags: + default: false + description: |- + PreserveSourceTags copies each source VM's vSphere tags and custom attributes + to the migrated VM as instance metadata. Applies to all VMs in the plan. + type: boolean vmMigrationPlans: description: VMMigrationPlans is the reference to the VM migration plan @@ -3251,6 +3658,12 @@ spec: cpu: description: CPU is the number of CPUs in the virtual machine type: integer + customAttributes: + additionalProperties: + type: string + description: CustomAttributes maps vSphere custom attribute names + to their values for the VM + type: object datastores: description: Datastores is the list of datastores for the virtual machine @@ -3358,6 +3771,13 @@ spec: items: type: string type: array + tags: + additionalProperties: + type: string + description: |- + Tags maps vSphere tag category names to a comma-separated list of the + tag names attached to the VM in that category (e.g. "env" -> "production") + type: object vmState: description: VMState is the state of the virtual machine type: string @@ -3915,6 +4335,44 @@ rules: --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole +metadata: + labels: + app.kubernetes.io/managed-by: kustomize + app.kubernetes.io/name: migration + name: migration-migrationblueprint-editor-role +rules: +- apiGroups: + - vjailbreak.k8s.pf9.io + resources: + - migrationblueprints + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + labels: + app.kubernetes.io/managed-by: kustomize + app.kubernetes.io/name: migration + name: migration-migrationblueprint-viewer-role +rules: +- apiGroups: + - vjailbreak.k8s.pf9.io + resources: + - migrationblueprints + verbs: + - get + - list + - watch +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole metadata: labels: app.kubernetes.io/managed-by: kustomize @@ -4863,6 +5321,7 @@ rules: - esxisshcreds - clustermigrations - esximigrations + - migrationblueprints - migrationplans - migrations - migrationtemplates @@ -4953,4 +5412,37 @@ subjects: - kind: ServiceAccount name: ui-manager-sa namespace: migration-system + +--- +apiVersion: v1 +kind: ServiceAccount +metadata: + name: vjailbreak-ai-sa + namespace: migration-system +--- +apiVersion: v1 +kind: PersistentVolumeClaim +metadata: + name: vjailbreak-ai-data + namespace: migration-system +spec: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi +--- +apiVersion: v1 +kind: Service +metadata: + name: vjailbreak-ai + namespace: migration-system +spec: + selector: + app: vjailbreak-ai + type: ClusterIP + ports: + - protocol: TCP + port: 8080 + targetPort: 8080 --- \ No newline at end of file diff --git a/deploy/05controller-deployment.yaml b/deploy/05controller-deployment.yaml index 813c1ff..31a5bdc 100644 --- a/deploy/05controller-deployment.yaml +++ b/deploy/05controller-deployment.yaml @@ -42,7 +42,7 @@ spec: - configMapRef: name: pf9-env optional: true - image: quay.io/platform9/vjailbreak-controller:v0.4.7 + image: quay.io/platform9/vjailbreak-controller:v0.4.8 imagePullPolicy: IfNotPresent lifecycle: preStop: diff --git a/deploy/06vpwned-deployment.yaml b/deploy/06vpwned-deployment.yaml index 0f2a756..162f6c2 100644 --- a/deploy/06vpwned-deployment.yaml +++ b/deploy/06vpwned-deployment.yaml @@ -40,7 +40,7 @@ spec: - configMapRef: name: pf9-env optional: true - image: quay.io/platform9/vjailbreak-vpwned:v0.4.7 + image: quay.io/platform9/vjailbreak-vpwned:v0.4.8 imagePullPolicy: IfNotPresent name: vpwned ports: diff --git a/deploy/07ui-deployment.yaml b/deploy/07ui-deployment.yaml index b956d36..ca2c41c 100644 --- a/deploy/07ui-deployment.yaml +++ b/deploy/07ui-deployment.yaml @@ -24,7 +24,7 @@ spec: serviceAccountName: ui-manager-sa containers: - name: vjailbreak-ui-container - image: quay.io/platform9/vjailbreak-ui:v0.4.7 + image: quay.io/platform9/vjailbreak-ui:v0.4.8 imagePullPolicy: IfNotPresent ports: - containerPort: 80 diff --git a/deploy/08vjailbreak-ai-deployment.yaml b/deploy/08vjailbreak-ai-deployment.yaml new file mode 100644 index 0000000..4208703 --- /dev/null +++ b/deploy/08vjailbreak-ai-deployment.yaml @@ -0,0 +1,58 @@ + +apiVersion: apps/v1 +kind: Deployment +metadata: + name: vjailbreak-ai + namespace: migration-system +spec: + replicas: 1 + selector: + matchLabels: + app: vjailbreak-ai + template: + metadata: + labels: + app: vjailbreak-ai + spec: + affinity: + nodeAffinity: + requiredDuringSchedulingIgnoredDuringExecution: + nodeSelectorTerms: + - matchExpressions: + - key: node-role.kubernetes.io/control-plane + operator: Exists + serviceAccountName: vjailbreak-ai-sa + containers: + - name: vjailbreak-ai + image: quay.io/platform9/vjailbreak-ai:v0.4.8 + imagePullPolicy: IfNotPresent + ports: + - containerPort: 8080 + env: + - name: ANTHROPIC_API_KEY + valueFrom: + secretKeyRef: + name: vjailbreak-ai-secret + key: api-key + optional: true + - name: CHROMA_PATH + value: /data/chroma + - name: CONTEXT_PATH + value: /data/context.md + - name: ALLOWED_ORIGINS + value: "http://vjailbreak-ui-service.migration-system.svc.cluster.local" + volumeMounts: + - name: ai-data + mountPath: /data + resources: + requests: + memory: "256Mi" + cpu: "100m" + limits: + memory: "1Gi" + cpu: "500m" + volumes: + - name: ai-data + persistentVolumeClaim: + claimName: vjailbreak-ai-data +--- \ No newline at end of file diff --git a/deploy/installer.yaml b/deploy/installer.yaml index 7bb0788..619077b 100644 --- a/deploy/installer.yaml +++ b/deploy/installer.yaml @@ -878,6 +878,356 @@ spec: --- apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.17.1 + name: migrationblueprints.vjailbreak.k8s.pf9.io +spec: + group: vjailbreak.k8s.pf9.io + names: + kind: MigrationBlueprint + listKind: MigrationBlueprintList + plural: migrationblueprints + singular: migrationblueprint + scope: Namespaced + versions: + - additionalPrinterColumns: + - jsonPath: .spec.displayName + name: Display Name + type: string + - jsonPath: .spec.vmwareRef + name: Source + type: string + - jsonPath: .spec.pcdRef + name: Destination + type: string + - jsonPath: .metadata.creationTimestamp + name: Age + type: date + name: v1alpha1 + schema: + openAPIV3Schema: + description: |- + MigrationBlueprint is the Schema for the migrationblueprints API. It stores a + reusable, named migration configuration that the UI presents as a "Migration + Template": users save a configuration once and apply it later to pre-populate + the migration form. Blueprints are not read by the migration controller. + properties: + apiVersion: + description: |- + APIVersion defines the versioned schema of this representation of an object. + Servers should convert recognized schemas to the latest internal value, and + may reject unrecognized values. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources + type: string + kind: + description: |- + Kind is a string value representing the REST resource this object represents. + Servers may infer this from the endpoint the client submits requests to. + Cannot be updated. + In CamelCase. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds + type: string + metadata: + type: object + spec: + description: |- + MigrationBlueprintSpec defines a reusable, named snapshot of a migration + configuration. It captures everything from the migration form except the VM + selection, so the UI can present it as a "Migration Template" and pre-fill + the migration wizard. Blueprints are consumed only by the UI: applying one + still creates the regular MigrationTemplate/MigrationPlan/mapping resources, + so the migration controller never reads a blueprint. + properties: + advancedOptions: + description: AdvancedOptions captures the advanced migration options + to pre-fill + properties: + acknowledgeNetworkConflictRisk: + description: AcknowledgeNetworkConflictRisk indicates that the + user acknowledges the risk of network conflicts when doing live + migration + type: boolean + granularNetworks: + description: GranularNetworks is a list of networks to be migrated + items: + type: string + type: array + granularPorts: + description: GranularPorts is a list of ports to be migrated + items: + type: string + type: array + granularVolumeTypes: + description: GranularVolumeTypes is a list of volume types to + be migrated + items: + type: string + type: array + imageProfiles: + description: ImageProfiles is the ordered list of VolumeImageProfile + names to apply to the migrated VM's boot volume. + items: + type: string + type: array + networkPersistence: + description: NetworkPersistence instructs the migration helper + to persist the source networking configuration + type: boolean + periodicSyncEnabled: + description: PeriodicSyncEnabled is a boolean to enable periodic + sync + type: boolean + periodicSyncInterval: + description: PeriodicSyncInterval is the interval at which the + migration plan should be synced + type: string + removeVMwareTools: + default: true + description: |- + RemoveVMwareTools instructs the migration helper to remove VMware Tools post migration. + Defaults to true since most migrations require VMware Tools removal. + type: boolean + type: object + arrayCredsMappings: + description: |- + ArrayCredsMappings is an inline snapshot of datastore-to-ArrayCreds pairs. + Used when StorageCopyMethod is "StorageAcceleratedCopy". + items: + description: DatastoreArrayCredsMapping represents a mapping between + a VMware datastore and ArrayCreds + properties: + source: + description: Source is the name of the source datastore in VMware + type: string + target: + description: Target is the name of the ArrayCreds resource to + use for this datastore + type: string + required: + - source + - target + type: object + type: array + customMetadata: + additionalProperties: + type: string + description: |- + CustomMetadata is a map of additional key-value pairs applied as instance + metadata to every migrated VM + type: object + dataOnly: + description: |- + DataOnly copies and converts disks to Cinder volumes without creating an + OpenStack VM + type: boolean + description: + description: Description is a free-text description of the blueprint + type: string + displayName: + description: DisplayName is the human-readable name of the blueprint + shown in the UI + minLength: 1 + type: string + fallbackToDHCP: + description: FallbackToDHCP falls back to DHCP when static IP assignment + is not possible + type: boolean + firstBootScript: + description: FirstBootScript is the script to run on first boot of + migrated VMs + type: string + migrationStrategy: + description: |- + MigrationStrategy captures the migration type, cutover windows, and + health-check settings to pre-fill into the migration form. + Optional so partially filled forms can be saved as templates. + properties: + adminInitiatedCutOver: + default: false + type: boolean + arrayOffload: + default: false + type: boolean + dataCopyStart: + format: date-time + type: string + dataOnly: + default: false + description: |- + DataOnly skips OpenStack VM creation; converted Cinder volumes are left staged. + Compatible with all strategy types (hot, cold, mock). + type: boolean + disconnectSourceNetwork: + default: false + type: boolean + healthCheckPort: + default: "443" + type: string + performHealthChecks: + default: false + type: boolean + type: + enum: + - hot + - cold + - mock + type: string + vmCutoverEnd: + format: date-time + type: string + vmCutoverStart: + format: date-time + type: string + required: + - type + type: object + networkMappings: + description: |- + NetworkMappings is an inline snapshot of source-to-target network pairs. + Copied by value rather than referencing a NetworkMapping resource, since + those are per-migration objects that may be mutated or deleted. + items: + description: Network represents a mapping between source and target + networks + properties: + source: + description: Source is the name of the source network in VMware + type: string + target: + description: Target is the name of the target network in OpenStack + type: string + required: + - source + - target + type: object + type: array + noVMwareClusterFilter: + description: |- + NoVMwareClusterFilter records that the user explicitly chose the + "No Cluster" source filter: VMs on standalone ESXi hosts that are not + part of any vCenter cluster in the datacenter referenced by VMwareRef. + This is distinct from VMwareClusterName simply not being set yet, so it + is tracked as its own field rather than inferred from an empty string. + type: boolean + osFamily: + description: OSFamily is the OS type of the virtual machines this + blueprint targets + enum: + - windowsGuest + - linuxGuest + type: string + pcdRef: + description: |- + PCDRef is the name of the OpenstackCreds to use as the destination PCD environment. + Optional so partially filled forms can be saved as templates. + type: string + postMigrationAction: + description: PostMigrationAction captures the post-migration actions + to pre-fill + properties: + folderName: + type: string + moveToFolder: + type: boolean + renameVm: + type: boolean + suffix: + type: string + type: object + preserveSourceTags: + description: |- + PreserveSourceTags copies each source VM's vSphere tags and custom + attributes to the migrated VM as instance metadata + type: boolean + proxyVMRef: + description: |- + ProxyVMRef references the ProxyVM to use for data copy. + Used when StorageCopyMethod is "HotAdd". + properties: + name: + default: "" + description: |- + Name of the referent. + This field is effectively required, but due to backwards compatibility is + allowed to be empty. Instances of this type with an empty value here are + almost certainly wrong. + More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + type: string + type: object + x-kubernetes-map-type: atomic + securityGroups: + description: SecurityGroups is the list of OpenStack security group + names to apply + items: + type: string + type: array + serverGroup: + description: ServerGroup is the OpenStack server group to place migrated + VMs into + type: string + storageCopyMethod: + default: normal + description: StorageCopyMethod indicates the method to use for storage + migration + enum: + - normal + - StorageAcceleratedCopy + - HotAdd + type: string + storageMappings: + description: |- + StorageMappings is an inline snapshot of source-to-target storage pairs. + Copied by value for the same reason as NetworkMappings. + items: + description: Storage represents a mapping between source and target + storage types + properties: + source: + description: Source is the name of the source storage type in + VMware + type: string + target: + description: Target is the name of the target storage type in + OpenStack + type: string + required: + - source + - target + type: object + type: array + targetPCDClusterName: + description: TargetPCDClusterName is the name of the PCD cluster to + migrate VMs into + type: string + useGPUFlavor: + description: UseGPUFlavor indicates if the migration should filter + and use GPU-enabled flavors + type: boolean + virtioWinDriver: + description: VirtioWinDriver is the virtio-win driver version to use + for Windows guests + type: string + vmwareClusterName: + description: |- + VMwareClusterName is the name of the source vCenter cluster VMs are + migrated from. + type: string + vmwareRef: + description: |- + VMwareRef is the name of the VMwareCreds to use as the source environment. + Optional so partially filled forms can be saved as templates. + type: string + required: + - displayName + type: object + type: object + served: true + storage: true + subresources: {} +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition metadata: annotations: controller-gen.kubebuilder.io/version: v0.17.1 @@ -976,10 +1326,20 @@ spec: migration plan should be synced type: string removeVMwareTools: - description: RemoveVMwareTools instructs the migration helper - to remove VMware Tools post migration + default: true + description: |- + RemoveVMwareTools instructs the migration helper to remove VMware Tools post migration. + Defaults to true since most migrations require VMware Tools removal. type: boolean type: object + customMetadata: + additionalProperties: + type: string + description: |- + CustomMetadata is a map of additional key-value pairs applied as instance + metadata to every migrated VM in the plan. Keys here override colliding + keys derived from preserved source tags. + type: object fallbackToDHCP: type: boolean firstBootScript: @@ -998,6 +1358,12 @@ spec: dataCopyStart: format: date-time type: string + dataOnly: + default: false + description: |- + DataOnly skips OpenStack VM creation; converted Cinder volumes are left staged. + Compatible with all strategy types (hot, cold, mock). + type: boolean disconnectSourceNetwork: default: false type: boolean @@ -1069,6 +1435,12 @@ spec: suffix: type: string type: object + preserveSourceTags: + default: false + description: |- + PreserveSourceTags copies each source VM's vSphere tags and custom attributes + to the migrated VM as instance metadata. Applies to all VMs in the plan. + type: boolean securityGroups: items: type: string @@ -1169,6 +1541,10 @@ spec: spec: description: Spec defines the desired state of Migration properties: + dataOnly: + description: DataOnly indicates no OpenStack VM should be created + after disk conversion. + type: boolean disconnectSourceNetwork: description: |- DisconnectSourceNetwork specifies whether to disconnect the source VM's network interfaces @@ -1202,6 +1578,12 @@ spec: podRef: description: PodRef is the name of the pod type: string + preserveSourceTags: + default: false + description: |- + PreserveSourceTags indicates whether the source VM's vSphere tags and custom + attributes are copied to the migrated VM as instance metadata. + type: boolean vmName: description: VMName is the name of the VM getting migrated from VMWare to Openstack @@ -1297,6 +1679,7 @@ spec: - IdentifyingBlockDevices - HotAddTransferInProgress - HotAddCleanup + - DataCopied type: string retryable: description: |- @@ -1304,6 +1687,12 @@ spec: Set to false for VMs with RDM (Raw Device Mapping) disks that share storage, as RDM disk migration state prevents automatic retry. type: boolean + stagedVolumeIDs: + description: StagedVolumeIDs lists the Cinder volume IDs created during + a data-only migration. + items: + type: string + type: array syncWarningMessage: description: |- SyncWarningMessage contains the warning message when periodic sync is in a warning state. @@ -1440,10 +1829,6 @@ spec: description: TargetPCDClusterName is the name of the PCD cluster where the virtual machine will be migrated type: string - useFlavorless: - description: UseFlavorless indicates if the migration should use flavorless - VM creation for PCD. - type: boolean useGPUFlavor: description: UseGPUFlavor indicates if the migration should filter and use GPU-enabled flavors. @@ -2410,8 +2795,10 @@ spec: migration plan should be synced type: string removeVMwareTools: - description: RemoveVMwareTools instructs the migration helper - to remove VMware Tools post migration + default: true + description: |- + RemoveVMwareTools instructs the migration helper to remove VMware Tools post migration. + Defaults to true since most migrations require VMware Tools removal. type: boolean type: object bmConfigRef: @@ -2509,6 +2896,14 @@ spec: - vmSequence type: object type: array + customMetadata: + additionalProperties: + type: string + description: |- + CustomMetadata is a map of additional key-value pairs applied as instance + metadata to every migrated VM in the plan. Keys here override colliding + keys derived from preserved source tags. + type: object firstBootScript: default: echo "Add your startup script here!" type: string @@ -2525,6 +2920,12 @@ spec: dataCopyStart: format: date-time type: string + dataOnly: + default: false + description: |- + DataOnly skips OpenStack VM creation; converted Cinder volumes are left staged. + Compatible with all strategy types (hot, cold, mock). + type: boolean disconnectSourceNetwork: default: false type: boolean @@ -2566,6 +2967,12 @@ spec: suffix: type: string type: object + preserveSourceTags: + default: false + description: |- + PreserveSourceTags copies each source VM's vSphere tags and custom attributes + to the migrated VM as instance metadata. Applies to all VMs in the plan. + type: boolean vmMigrationPlans: description: VMMigrationPlans is the reference to the VM migration plan @@ -3251,6 +3658,12 @@ spec: cpu: description: CPU is the number of CPUs in the virtual machine type: integer + customAttributes: + additionalProperties: + type: string + description: CustomAttributes maps vSphere custom attribute names + to their values for the VM + type: object datastores: description: Datastores is the list of datastores for the virtual machine @@ -3358,6 +3771,13 @@ spec: items: type: string type: array + tags: + additionalProperties: + type: string + description: |- + Tags maps vSphere tag category names to a comma-separated list of the + tag names attached to the VM in that category (e.g. "env" -> "production") + type: object vmState: description: VMState is the state of the virtual machine type: string @@ -3915,6 +4335,44 @@ rules: --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole +metadata: + labels: + app.kubernetes.io/managed-by: kustomize + app.kubernetes.io/name: migration + name: migration-migrationblueprint-editor-role +rules: +- apiGroups: + - vjailbreak.k8s.pf9.io + resources: + - migrationblueprints + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + labels: + app.kubernetes.io/managed-by: kustomize + app.kubernetes.io/name: migration + name: migration-migrationblueprint-viewer-role +rules: +- apiGroups: + - vjailbreak.k8s.pf9.io + resources: + - migrationblueprints + verbs: + - get + - list + - watch +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole metadata: labels: app.kubernetes.io/managed-by: kustomize @@ -4741,7 +5199,7 @@ spec: - configMapRef: name: pf9-env optional: true - image: quay.io/platform9/vjailbreak-controller:v0.4.7 + image: quay.io/platform9/vjailbreak-controller:v0.4.8 imagePullPolicy: IfNotPresent lifecycle: preStop: @@ -4848,7 +5306,7 @@ spec: - configMapRef: name: pf9-env optional: true - image: quay.io/platform9/vjailbreak-vpwned:v0.4.7 + image: quay.io/platform9/vjailbreak-vpwned:v0.4.8 imagePullPolicy: IfNotPresent name: vpwned ports: @@ -4977,7 +5435,7 @@ spec: serviceAccountName: ui-manager-sa containers: - name: vjailbreak-ui-container - image: quay.io/platform9/vjailbreak-ui:v0.4.7 + image: quay.io/platform9/vjailbreak-ui:v0.4.8 imagePullPolicy: IfNotPresent ports: - containerPort: 80 @@ -5130,6 +5588,7 @@ rules: - esxisshcreds - clustermigrations - esximigrations + - migrationblueprints - migrationplans - migrations - migrationtemplates @@ -5220,3 +5679,93 @@ subjects: - kind: ServiceAccount name: ui-manager-sa namespace: migration-system + +--- +apiVersion: v1 +kind: ServiceAccount +metadata: + name: vjailbreak-ai-sa + namespace: migration-system +--- +apiVersion: v1 +kind: PersistentVolumeClaim +metadata: + name: vjailbreak-ai-data + namespace: migration-system +spec: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: vjailbreak-ai + namespace: migration-system +spec: + replicas: 1 + selector: + matchLabels: + app: vjailbreak-ai + template: + metadata: + labels: + app: vjailbreak-ai + spec: + affinity: + nodeAffinity: + requiredDuringSchedulingIgnoredDuringExecution: + nodeSelectorTerms: + - matchExpressions: + - key: node-role.kubernetes.io/control-plane + operator: Exists + serviceAccountName: vjailbreak-ai-sa + containers: + - name: vjailbreak-ai + image: quay.io/platform9/vjailbreak-ai:v0.4.8 + imagePullPolicy: IfNotPresent + ports: + - containerPort: 8080 + env: + - name: ANTHROPIC_API_KEY + valueFrom: + secretKeyRef: + name: vjailbreak-ai-secret + key: api-key + optional: true + - name: CHROMA_PATH + value: /data/chroma + - name: CONTEXT_PATH + value: /data/context.md + - name: ALLOWED_ORIGINS + value: "http://vjailbreak-ui-service.migration-system.svc.cluster.local" + volumeMounts: + - name: ai-data + mountPath: /data + resources: + requests: + memory: "256Mi" + cpu: "100m" + limits: + memory: "1Gi" + cpu: "500m" + volumes: + - name: ai-data + persistentVolumeClaim: + claimName: vjailbreak-ai-data +--- +apiVersion: v1 +kind: Service +metadata: + name: vjailbreak-ai + namespace: migration-system +spec: + selector: + app: vjailbreak-ai + type: ClusterIP + ports: + - protocol: TCP + port: 8080 + targetPort: 8080 diff --git a/deploy/vjailbreak-ai-context-configmap.yaml b/deploy/vjailbreak-ai-context-configmap.yaml new file mode 100644 index 0000000..cc2d980 --- /dev/null +++ b/deploy/vjailbreak-ai-context-configmap.yaml @@ -0,0 +1,75 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: vjailbreak-ai-context + namespace: migration-system +data: + additional_context: | + ## vJailbreak Architecture + + vJailbreak migrates VMware VMs to OpenStack in this sequence: + 1. Controller creates a v2v-helper pod per migration + 2. v2v-helper runs virt-v2v with the nbdkit+VDDK plugin to stream VMDK data + 3. Converted disk is uploaded to OpenStack Glance as QCOW2 (or written to a Cinder volume via RAW) + 4. Nova boots the migrated VM using the uploaded image or volume + + Key components: + - **migration-controller-manager** — Kubernetes controller in `migration-system` namespace; reconciles Migration CRs + - **v2v-helper pod** — named `` in `migration-system`; runs virt-v2v; one pod per Migration + - **VDDK** — VMware Virtual Disk Development Kit at `/home/ubuntu/vmware-vix-disklib-distrib/` on the vJailbreak VM + - **Conversion scratch space** — `emptyDir` at `/var/tmp/v2v-conversion-disks/` inside v2v-helper pod (~2× VM disk size required) + - **storageCopyMethod** — `hot` (live/CBT incremental) or `cold` (offline full-copy); set in MigrationTemplate + + CRD hierarchy: Migration → MigrationPlan → MigrationTemplate → NetworkMapping + StorageMapping + + --- + + ## Common Failure Patterns and Fixes + + ### VDDK / VMware Connectivity + | Symptom | Likely Cause | Fix | + |---------|-------------|-----| + | `VixDiskLib_Open failed` / `Transport error` | VDDK library missing or wrong version | Verify `/home/ubuntu/vmware-vix-disklib-distrib/` exists; VDDK 7.0+ required | + | `Failed to connect to host ` | ESXi hostname not resolving in pod | Add ESXi host entries to `/etc/hosts` on vJailbreak VM, then restart controller | + | `SSL certificate verify failed` | vCenter TLS cert untrusted | Set `InsecureSkipVerify: true` in VMwareCreds or add cert to trust store | + | `NoPermission` / `NotAuthenticated` | Insufficient vCenter permissions | Account needs Datastore.Browse, VirtualMachine.Provisioning.DiskRandomRead, Global.DisableMethods | + | `CBT not enabled` / `queryChangedDiskAreas` error | Changed Block Tracking disabled on source VM | Enable CBT in vCenter VM settings, create a snapshot, then retry | + + ### virt-v2v / Conversion + | Symptom | Likely Cause | Fix | + |---------|-------------|-----| + | `No space left on device` | Node disk exhausted during conversion | Ensure node has ≥2× VM disk size free in `/var/tmp`; check `df -h` on vJailbreak VM | + | `libguestfs: error: could not create appliance` | libguestfs kernel/QEMU unavailable in pod | Check pod for missing KVM device; vJailbreak VM must have nested virt or bare-metal KVM | + | `Failed to locate virtio drivers` / `virtio-win` | virtio-win ISO not found | Confirm virtio-win ISO is present at the expected path in the v2v-helper image | + | `qemu-img: Could not open` / `corrupt` | VMDK corrupt or CBT snapshot stale | Remove stale snapshots in vCenter; try `storageCopyMethod: cold` | + | `virt-v2v: error: inspection of disk image failed` | Unsupported guest OS | Check virt-v2v support page; Windows requires virtio-win, Linux needs grub accessible | + | Conversion stuck / timeout | Very large VM or slow storage | Increase `migrationTimeout` in MigrationPlan; check network throughput between vJailbreak and ESXi | + + ### OpenStack Upload / Deploy + | Symptom | Likely Cause | Fix | + |---------|-------------|-----| + | `No valid host found` | Flavor constraints or resource exhaustion | Verify flavor exists and Nova has capacity; check `openstack flavor list` | + | `Quota exceeded` | OpenStack project quota hit | Check `openstack quota show` for instances, cores, RAM, volumes, gigabytes | + | `Network could not be found` | NetworkMapping source or target name mismatch | Verify NetworkMapping.Spec matches exact OpenStack network names | + | `VolumeType not found` | StorageMapping mismatch | Verify StorageMapping.Spec matches exact Cinder volume type names; case-sensitive | + | `Provided image is too large` | Glance upload size limit | Increase `glance-api.conf image_size_cap`; or use `storageCopyMethod: raw-volume` to bypass Glance | + | `ImageNotFound` during Nova boot | Glance upload failed or image in error state | Check `openstack image list`; look for image in `killed` or `queued` state | + + ### Kubernetes / Controller + | Symptom | Likely Cause | Fix | + |---------|-------------|-----| + | Pod `OOMKilled` | Insufficient memory for conversion | Large Windows VMs need 4–8 GB; increase v2v-helper memory limit in MigrationPlan | + | Pod stuck in `Pending` | Node selector mismatch or node resource shortage | Check `kubectl describe pod ` for scheduling events | + | `context deadline exceeded` in controller | Migration timeout too short | Increase `migrationTimeout` in MigrationPlan spec | + | Migration stuck in `DataCopyStart` | NBD/VDDK stream stalled | Check v2v-helper logs for `nbdkit` errors; restart migration | + + --- + + ## Analysis Guidelines + + - **Always quote specific log lines** as evidence in `root_cause` — vague causes are not actionable + - **fix_steps** should be concrete shell commands or UI steps, not generic advice + - Check `storageCopyMethod` (`hot` vs `cold`) — hot migrations have extra pre-copy phases that can fail + - If `phase` is `Failed` but logs are empty, check controller logs for reconcile errors mentioning the Migration name + - Network and storage mapping errors almost always have the exact mismatched name in the log — quote it + - If confidence is `low` or `none`, list the specific information that would be needed to diagnose further diff --git a/deploy/vjailbreak-ai/context-configmap.yaml b/deploy/vjailbreak-ai/context-configmap.yaml new file mode 100644 index 0000000..42654b8 --- /dev/null +++ b/deploy/vjailbreak-ai/context-configmap.yaml @@ -0,0 +1,19 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: vjailbreak-ai-context + namespace: migration-system +data: + # Operator-provided context injected into every AI analysis prompt. + # Edit the value below to add site-specific information. + # Examples: + # - Custom VDDK install path + # - Known ESXi certificate issues in this environment + # - Network restrictions that affect migrations + # - OpenStack volume type mappings in use + additional_context: | + # Add operator-specific notes here. + # Example: + # VDDK is installed at /opt/vmware/vddk (non-standard path). + # All ESXi hosts use self-signed TLS certificates -- SSL errors are expected. + # OpenStack Cinder volume type "ssd" maps to VMware datastore "fast-ssd". diff --git a/deploy/vjailbreak-ai/deployment.yaml b/deploy/vjailbreak-ai/deployment.yaml new file mode 100644 index 0000000..a421ba7 --- /dev/null +++ b/deploy/vjailbreak-ai/deployment.yaml @@ -0,0 +1,72 @@ +# Prerequisites: +# Create the API key Secret via Settings → AI in the vJailbreak UI, or manually: +# kubectl -n migration-system create secret generic vjailbreak-ai-secret \ +# --from-literal=api-key= \ +# --from-literal=admin-key= +# The Deployment uses optional: true so it starts without the Secret, +# but AI analysis calls will fail until the key is configured. +apiVersion: apps/v1 +kind: Deployment +metadata: + name: vjailbreak-ai + namespace: migration-system + labels: + app: vjailbreak-ai +spec: + replicas: 1 + selector: + matchLabels: + app: vjailbreak-ai + template: + metadata: + labels: + app: vjailbreak-ai + spec: + securityContext: + runAsNonRoot: true + runAsUser: 1001 + fsGroup: 1001 + containers: + - name: vjailbreak-ai + image: quay.io/platform9/vjailbreak-ai:main + ports: + - containerPort: 8080 + env: + - name: ANTHROPIC_API_KEY + valueFrom: + secretKeyRef: + name: vjailbreak-ai-secret + key: api-key + optional: true # Secret must be created via Settings UI before AI analysis works + - name: ADMIN_API_KEY + valueFrom: + secretKeyRef: + name: vjailbreak-ai-secret + key: admin-key + optional: true + volumeMounts: + - name: chroma-data + mountPath: /data + resources: + requests: + cpu: 200m + memory: 512Mi + limits: + cpu: "1" + memory: 1Gi + livenessProbe: + httpGet: + path: /health + port: 8080 + initialDelaySeconds: 30 + periodSeconds: 30 + readinessProbe: + httpGet: + path: /health + port: 8080 + initialDelaySeconds: 10 + periodSeconds: 10 + volumes: + - name: chroma-data + persistentVolumeClaim: + claimName: vjailbreak-ai-chroma diff --git a/deploy/vjailbreak-ai/pvc.yaml b/deploy/vjailbreak-ai/pvc.yaml new file mode 100644 index 0000000..1ab40c8 --- /dev/null +++ b/deploy/vjailbreak-ai/pvc.yaml @@ -0,0 +1,11 @@ +apiVersion: v1 +kind: PersistentVolumeClaim +metadata: + name: vjailbreak-ai-chroma + namespace: migration-system +spec: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi diff --git a/deploy/vjailbreak-ai/service.yaml b/deploy/vjailbreak-ai/service.yaml new file mode 100644 index 0000000..0121f7c --- /dev/null +++ b/deploy/vjailbreak-ai/service.yaml @@ -0,0 +1,12 @@ +apiVersion: v1 +kind: Service +metadata: + name: vjailbreak-ai + namespace: migration-system +spec: + selector: + app: vjailbreak-ai + ports: + - port: 8080 + targetPort: 8080 + type: ClusterIP diff --git a/image_builder/configs/daemonset.yaml b/image_builder/configs/daemonset.yaml index 4c38cd4..84d6a24 100644 --- a/image_builder/configs/daemonset.yaml +++ b/image_builder/configs/daemonset.yaml @@ -59,21 +59,56 @@ spec: - | IS_MASTER=$(grep -E '^export IS_MASTER=' /etc/pf9/k3s.env | cut -d'=' -f2) MASTER_IP=$(grep -E '^export MASTER_IP=' /etc/pf9/k3s.env | cut -d'=' -f2) + + # Migration considered idle if quiet for this long; only then is it eligible for cleanup. + ACTIVITY_WINDOW_MIN=60 + # Delete threshold: older than 24h. + RETENTION_MTIME=+1 + + # Deletes migration--.log + migration--/ as one unit, + # skipping the pair if either was written to within ACTIVITY_WINDOW_MIN. + cleanup_dir() { + local base_dir="$1" + [ -d "$base_dir" ] || return 0 + + local names + names=$( { find "$base_dir" -maxdepth 1 -type f -name 'migration-*.log' -printf '%f\n' 2>/dev/null | sed 's/\.log$//' + find "$base_dir" -mindepth 1 -maxdepth 1 -type d -name 'migration-*' -printf '%f\n' 2>/dev/null + } | sort -u ) + + echo "$names" | while IFS= read -r name; do + [ -n "$name" ] || continue + local log_file="$base_dir/$name.log" + local dir="$base_dir/$name" + + if find "$log_file" "$dir" -type f -mmin -"$ACTIVITY_WINDOW_MIN" 2>/dev/null | grep -q .; then + echo "[cleanup] Skipping $name (active within last ${ACTIVITY_WINDOW_MIN}m)" + continue + fi + + [ -f "$log_file" ] && find "$log_file" -mtime "$RETENTION_MTIME" -delete 2>/dev/null + if [ -d "$dir" ]; then + find "$dir" -type f -mtime "$RETENTION_MTIME" -delete 2>/dev/null + rmdir "$dir" 2>/dev/null || true + fi + done + + # Stray non-migration files: clean by age alone. + find "$base_dir" -maxdepth 1 -type f ! -name 'migration-*.log' -mtime "$RETENTION_MTIME" -delete 2>/dev/null || true + } + if [ "$IS_MASTER" = "true" ]; then echo "Master node detected. IP: $MASTER_IP" - - # Clean up logs older than 24 hours on master node - echo "Cleaning up logs older than 24 hours on master node" - # Note: Not restarting services to avoid disruption - # Instead, properly closing files in the v2v-helper code - # Clean master's own logs and force delete any stuck files - find /var/log/pf9 -type f -mtime +1 -delete 2>/dev/null || true - # Clean worker logs stored on master and force delete any stuck files - find /var/lib/rsync/worker-logs -type f -mtime +1 -delete 2>/dev/null || true - # Clear deleted but open files (useful when processes still hold file handles) - echo "Clearing deleted but open files..." - lsof +L1 | grep "/var/log/pf9\|/var/lib/rsync/worker-logs" | awk '{print $2}' | xargs -r kill -HUP 2>/dev/null || true - + + master_cleanup_loop() { + while true; do + echo "Cleaning up logs older than 24 hours on master node ($(date))" + cleanup_dir /var/log/pf9 + sleep 300 + done + } + master_cleanup_loop & + echo "Starting rsync daemon..." rsync --daemon --config=/etc/pf9/rsyncd.conf --no-detach if ps aux | grep rsync | grep -v grep; then @@ -84,29 +119,24 @@ spec: fi else echo "Worker node detected. Syncing from Master..." + + agent_cleanup_loop() { + while true; do + echo "Cleaning up logs older than 24 hours on worker node ($(date))" + cleanup_dir /var/log/pf9 + sync + echo 3 > /proc/sys/vm/drop_caches 2>/dev/null || true + sleep 300 + done + } + agent_cleanup_loop & + while true; do mkdir -p /home/ubuntu/vmware-vix-disklib-distrib/ - - # Clean up logs older than 24 hours on worker node - echo "Cleaning up logs older than 24 hours on worker node" - # Note: Not restarting services to avoid disruption - # Instead, properly closing files in the v2v-helper code - # Force delete old log files - find /var/log/pf9 -type f -mtime +1 -delete 2>/dev/null || true - # Clear deleted but open files - echo "Clearing deleted but open files..." - lsof +L1 | grep "/var/log/pf9" | awk '{print $2}' | xargs -r kill -HUP 2>/dev/null || true - # Free up disk space by clearing caches - sync - echo 3 > /proc/sys/vm/drop_caches 2>/dev/null || true - + echo "Syncing from rsync://${MASTER_IP}" rsync -avz rsync://${MASTER_IP}/vmwarelib/ /home/ubuntu/vmware-vix-disklib-distrib/ - - # Sync logs from worker to master rsync -avz /var/log/pf9 rsync://${MASTER_IP}/worker-logs - - # Sync virtio-win drivers from master to worker rsync -avz rsync://${MASTER_IP}/virtio-win /home/ubuntu/virtio-win if [ $? -eq 0 ]; then diff --git a/releases/v0.4.8.md b/releases/v0.4.8.md new file mode 100644 index 0000000..f5bb403 --- /dev/null +++ b/releases/v0.4.8.md @@ -0,0 +1,88 @@ +# v0.4.8 + +- **Tag:** `v0.4.8` +- **Published:** 2026-08-04T09:22:20Z +- **Source release:** https://github.com/platform9/vjailbreak/releases/tag/v0.4.8 + +## Release Notes + +## What's Changed +* #2067: Block/Grey out the already registered proxy vm, this could mismatch the secret by @AbhijeetThakur in https://github.com/platform9/vjailbreak/pull/2105 +* [2083] Infer hotplug intent from assigned flavor by @sarika-pf9 in https://github.com/platform9/vjailbreak/pull/2109 +* fix: security workflow fork PR comments + graphify git noise by @OmkarDeshpande7 in https://github.com/platform9/vjailbreak/pull/2116 +* test(v2v-helper): repair stale mkinitrd LVM wrapper unit tests by @gtherond in https://github.com/platform9/vjailbreak/pull/2075 +* [2106] Fix wrong stage shown for Hot-Add proxy migrations by @meghansh-pf9 in https://github.com/platform9/vjailbreak/pull/2117 +* [2085] Block Persist source network interfaces when mapped network subnet doesn't match VM IPs by @sarika-pf9 in https://github.com/platform9/vjailbreak/pull/2110 +* [2095] Serve debug bundle as .tar.gz archive instead of single .txt file by @sarika-pf9 in https://github.com/platform9/vjailbreak/pull/2112 +* [2113] Improved subnet mismatch warning and Persist IP disabled warning by @sarika-pf9 in https://github.com/platform9/vjailbreak/pull/2126 +* refactor: remove deprecated files and components, streamline package dependencies by @AbhijeetThakur in https://github.com/platform9/vjailbreak/pull/2114 +* Don't call openstackcreds per cluster to get tenant name by @spai-p9 in https://github.com/platform9/vjailbreak/pull/2057 +* #2087 : Add confirmation dialog for node reprovisioning by @AbhijeetThakur in https://github.com/platform9/vjailbreak/pull/2131 +* #2094: [UI] Update Controller Log Section Styling to Match Migration Details Logs by @AbhijeetThakur in https://github.com/platform9/vjailbreak/pull/2136 +* [2128] Show assigned flavor and disk size for VMs selected for migration by @sarika-pf9 in https://github.com/platform9/vjailbreak/pull/2137 +* [1930] Fix CentOS 7 conversion failure from e2fsck bitmap errors by @meghansh-pf9 in https://github.com/platform9/vjailbreak/pull/2134 +* Delete associated agent nodes when OpenstackCreds is deleted by @OmkarDeshpande7 in https://github.com/platform9/vjailbreak/pull/2119 +* #2125: Move Proxy VM Credentials Navigation from Credentials to Migration Menu by @AbhijeetThakur in https://github.com/platform9/vjailbreak/pull/2138 +* Refactor: ReservePortsForVM by @spai-p9 in https://github.com/platform9/vjailbreak/pull/2140 +* [2012] Fix SUSE GRUB Legacy device references post-migration by @meghansh-pf9 in https://github.com/platform9/vjailbreak/pull/2133 +* AI analysis of migration failures by @OmkarDeshpande7 in https://github.com/platform9/vjailbreak/pull/1998 +* [1801] Twice HOTPLUG_CPU_MAX/HOTPLUG_MEMORY_MAX metadata so migrated VMs can be resized on PCD by @sarika-pf9 in https://github.com/platform9/vjailbreak/pull/2127 +* [2167] fix(v2v-helper): Install vendored libnbd with libnbd-devel by @sarika-pf9 in https://github.com/platform9/vjailbreak/pull/2166 +* When Ip is empty and fallback to dhcp true ask pcd to assign ip via dhcp by @spai-p9 in https://github.com/platform9/vjailbreak/pull/2153 +* [2161] fix(ui): Global Settings Save button silently dead by @sarika-pf9 in https://github.com/platform9/vjailbreak/pull/2162 +* [428] Added ability to preserve tags, attributes, and custom metadata attached to the virtual machine by @sarika-pf9 in https://github.com/platform9/vjailbreak/pull/2146 +* [2115] Fix IP address not populating on port creation due to MAC address case mismatch by @sarika-pf9 in https://github.com/platform9/vjailbreak/pull/2160 +* [428] UI: Added ability to preserve tags, attributes, and custom metadata attached to the virtual machine by @sarika-pf9 in https://github.com/platform9/vjailbreak/pull/2145 +* [2096] Default removeVMwareTools to true by @OmkarDeshpande7 in https://github.com/platform9/vjailbreak/pull/2159 +* Make clean up both agents and vjb nodes to run as background and improve reliability by @spai-p9 in https://github.com/platform9/vjailbreak/pull/2155 +* Refactoring boot partition detection by @meghansh-pf9 in https://github.com/platform9/vjailbreak/pull/2168 +* Separate out nbd and virtv2v debug logs by @spai-p9 in https://github.com/platform9/vjailbreak/pull/2165 +https://github.com/platform9/vjailbreak/pull/2172 +* [2173] Remove beta tag from Storage Accelerated Copy by @meghansh-pf9 in https://github.com/platform9/vjailbreak/pull/2177 +* [2152] Add MigrationBlueprint CRD to support migration templates by @sarika-pf9 in https://github.com/platform9/vjailbreak/pull/2158 +* [2175] UI patch call for proxy vm retry by @meghansh-pf9 in https://github.com/platform9/vjailbreak/pull/2181 +* [2176] Fix tags toggle needing two clicks to enable by @sarika-pf9 in https://github.com/platform9/vjailbreak/pull/2183 +* [2184] Fix Migration Options toggles needing two clicks to enable by @sarika-pf9 in https://github.com/platform9/vjailbreak/pull/2185 +* #2120: Add Support for Migration Templates and Saved Configurations by @AbhijeetThakur in https://github.com/platform9/vjailbreak/pull/2180 +* Configure guest network incase of ubuntu for fallback to dhcp case by @spai-p9 in https://github.com/platform9/vjailbreak/pull/2182 +* [2178] fix(migration): disable persist network interfaces when Preserve IP is off and mapped to diff network by @sarika-pf9 in https://github.com/platform9/vjailbreak/pull/2189 +* Feature: Add error propogation to UI from debug logs for failures by @spai-p9 in https://github.com/platform9/vjailbreak/pull/2170 +* [2213] Fix migration stuck in ConvertingDisk after transient VM creation timeout by @OmkarDeshpande7 in https://github.com/platform9/vjailbreak/pull/2192 +* [2143] unique port names for multi-NIC VMs using network+subnet+vmname by @OmkarDeshpande7 in https://github.com/platform9/vjailbreak/pull/2147 +* [2191] Grid/list toggle wraps left instead of staying inline on Templates tooolbar by @sarika-pf9 in https://github.com/platform9/vjailbreak/pull/2199 +* [2193] Prevent checkbox selection column from being hidden via Manage Columns by @sarika-pf9 in https://github.com/platform9/vjailbreak/pull/2198 +* [2194] Fix: Converting Disk status should not show count of disks by @sarika-pf9 in https://github.com/platform9/vjailbreak/pull/2200 +* [2202] Fix: AI Analysis key configuration redirect to correct settings tab by @sarika-pf9 in https://github.com/platform9/vjailbreak/pull/2204 +* feat: add data-only migration mode (no VM creation) (#492) by @OmkarDeshpande7 in https://github.com/platform9/vjailbreak/pull/2171 +* Support encrypted volume types in pcd. by @spai-p9 in https://github.com/platform9/vjailbreak/pull/2208 +* [2201] Show migration failure messages on the UI by @sarika-pf9 in https://github.com/platform9/vjailbreak/pull/2205 +* [2203] Fix: Disable migration progress bar by @sarika-pf9 in https://github.com/platform9/vjailbreak/pull/2210 +* #2209: Step check marks doesnt disable if user disable the option by @AbhijeetThakur in https://github.com/platform9/vjailbreak/pull/2215 +* [2214] Fix: Controller/pod log search returns zero results for lines that exist by @sarika-pf9 in https://github.com/platform9/vjailbreak/pull/2221 +* [2217] Fix: Remove stale network/storage mappings when VMs are deselected by @sarika-pf9 in https://github.com/platform9/vjailbreak/pull/2222 +* feat(dev): add Claude Code agent definitions for vjailbreak workflows by @OmkarDeshpande7 in https://github.com/platform9/vjailbreak/pull/2223 +* [2195] Fix incorrect step timings on the migration detail page by @sarika-pf9 in https://github.com/platform9/vjailbreak/pull/2225 +* [2213] Fix: Migration phase permanently frozen at Failed from transient kubelet events by @sarika-pf9 in https://github.com/platform9/vjailbreak/pull/2229 +* [2234] Fix: remove VM selection from template creation and refactor pruning of mappings by @sarika-pf9 in https://github.com/platform9/vjailbreak/pull/2233 +* [CI] modify pre-release actions to not skip builds by @OmkarDeshpande7 in https://github.com/platform9/vjailbreak/pull/2239 + +## New Contributors +* @gtherond made their first contribution in https://github.com/platform9/vjailbreak/pull/2075 + +## Highlights + +* Introduced log analysis for failed migrations using Anthropic AI to help identify potential root causes and simplify troubleshooting. +* Added ability to preserve source vCenter VM tags, attributes, and custom metadata during migration. +* Promoted Storage Assisted Migration to General Availability (GA). +* Added support for Migration Templates and Saved Configurations to streamline repeated migration workflows. +* Added support for Data-Only migration mode, enabling data transfer without creating the destination VM. +* Added support for migrating KMS-encrypted source VMs. + +## Known Limitations + +* The **Data-Only** migration option is not preserved when a migration is retried using the **Retry** button. See [[Issue #2245](https://github.com/platform9/vjailbreak/issues/2245)] for details. +* Migrations of VMs running **RHEL 7.9** may intermittently encounter a disk index-related issue. Retrying the migration typically resolves the problem. +* When using the **Hotplug** flavor for migration, the destination VM's **Maximum CPU** and **Maximum Memory** values are set to twice the original VM's configured CPU and memory in vCenter. +* Migration of **SUSE Linux Enterprise Server (SLES)** and **SUSE Linux Enterprise Desktop (SLED)** systems using **Legacy GRUB 0.97** is not supported. + +**Full Changelog**: https://github.com/platform9/vjailbreak/compare/v0.4.7...v0.4.8