Volume 從AFF Node上Volume Move到Non-AFF Node之後無法更改efficiency設定

最近碰到一個問題, Volume從AFF的Node透過Volume Move 搬到Non-AFF Node之後, 發現更改efficiency Policy設定會碰到以下Error

ONTAP API Failed: Failed to modify efficiency configuration for volume "Volume_Name" of Vserver "SVM_Name": Cannot perform efficiency operations on deprioritized volume. To be able to perform efficiency operations, run "volume efficiency promote" command first.




開Case跟support確認後, Support給了一個類似的kb, 透過kb的方式就可以解決這個問題
https://kb.netapp.com/app/answers/answer_view/a_id/1098121


1.將Volume efficiency of

Cluster::> volume efficiency off –vserver SVM_Name -volume Volume_Name

2.更改Volume Policy

Cluster::> volume efficiency modify -vserver SVM_Name -volume Volume_Name -policy Default

3.Volume efficiency on

Cluster::> volume efficiency on -vserver SVM_Name -volume Volume_Name

之後就可以正常更改efficiency設定

更換NetApp 控制器

Clustered Data ONTAP或是ONTAP9如果是在HA的配置下，更換控制器的方式比7-mode簡單，不需要另外reassign disk，系統會自動偵測到新的控制器並且assign Disk

NAC-Node1 >正常的控制器
NAC-Node2 >有問題的控制器

1.在NAC-Node1 takeover NAC-Node2

2.更換NAC-Node2 硬體, 更換完之後先確認新的控制器的系統時間
a.在NAC-Node1 的系統中用指令date確認目前ONTAP系統時間

NAC::> date

b.在NAC-Node2 的LOADER用指令show date確認時間

LOADER> show date

c.如果NAC-Node2的時間與NAC-Node1不一致, 可在NAC-Node2的LOADER 用指令set data mm/dd/yyyy以及
set time hh:mm:ss 更改時間(GMT), 更改後可在LOADER 再次用show data確認是否一致

LOADER> set date mm/dd/yyyy
LOADER> set time hh:mm:ss

3.更換完NAC-Node2 的控制器後, 將NAC-Node2開到Waiting for giveback的狀態後, 可在NAC-Node1透過storage failover show指令看到新的NAC-Node2 的System ID

NAC::> storage failover show
Takeover          
Node           Partner        Possible State Description  
-------------- -------------- -------- -------------------------------------
NAC-Node1   NAC-Node2   false    System ID changed on partner (Old:
                                 111111111, New: 222222222), In
                                 takeover
NAC-Node2   NAC-Node1   -        Waiting for giveback
2 entries were displayed.

4.將NAC-Node2 giveback, 系統會確認是否要更新成新的System ID

NAC::> storage failover giveback -ofnode NAC-Node2

Warning: System ID changed on partner. Disk ownership will be updated with new
system ID. Do you want to continue? {y|n}: y

Info: Run the storage failover show-giveback command to check giveback status. 

5.可透過storage failover show確認狀態, giveback後node需要幾分鐘同步資訊

NAC::> storage failover show
Takeover          
Node           Partner        Possible State Description  
-------------- -------------- -------- -------------------------------------
NAC-Node1   NAC-Node2   true          System ID changed on partner (Old:
                                      111111111, New: 222222222),
                                      Connected to NAC-Node2, Partial
                                      giveback
NAC-Node2   NAC-Node1   true          System ID changed on local (Old:
                                      111111111, New: 222222222),
                                      Connected to NAC-Node1. Waiting
                                      for cluster applications to come
                                      online on the local node. Offline
                                      applications: mgmt, vldb, vifmgr,
                                      bcomd, crs.
2 entries were displayed.

6.同步完成,控制器更換完成

NAC::> storage failover show
Takeover          
Node           Partner        Possible State Description  
-------------- -------------- -------- -------------------------------------
NAC-Node1   NAC-Node2   true     Connected to NAC-Node2
NAC-Node2   NAC-Node1   true     Connected to NAC-Node1
2 entries were displayed.

7.因為NetApp 的License key會綁控制器的序號, 所以更換完控制器後必須在90天內跟NetApp Support申請新的License key, 並且更新ONTAP裡的License, 否則原本的一些需要License的功能會失效

參考資料:
https://library.netapp.com/ecm/ecm_get_file/ECMP1199896

NetApp Modules for Ansible

最近在測試Ansible, 看到NetApp有支援 Ansible 的module, 就來測試一下

測試環境:
Ubuntu18.0.4
Ansible 2.7
ONTAP 9.2 Simulator

1.安裝Ansible

sudo apt update 
sudo apt install python-pip -y 
sudo apt-add-repository --yes --update ppa:ansible/ansible 
sudo apt install ansible

2.確認Ansible版本

ansible --version

3.安裝netapp python modules

sudo pip install netapp-lib

4.在NetApp storage上啟用 http-enabled

Cluster:> set -priv advanced
Cluster:> system services web modify -http-enabled true

5.接下來使用na_ontap_svm module測試透過Playbook建立新的SVM
https://docs.ansible.com/ansible/2.7/modules/na_ontap_svm_module.html#na-ontap-svm-module

sudo vim createsvm.yml

---
- name: Netapp-ansible-test
  hosts: localhost
  vars:
      login: &login
      hostname: 192.168.0.100
      username: admin
      password: passw0rd
  tasks:
      - name: Create SVM
        na_ontap_svm:
          state: present
          name: SVM_ansible01
          root_volume: vol1
          root_volume_aggregate: data_aggr01
          root_volume_security_style: mixed
          hostname: "{{ hostname }}"
          username: "{{ username }}"
          password: "{{ password }}"

6.執行Playbook

ansible-playbook createsvm.yml

7.從NetApp System Manager可看到新的SVM已建立





參考資料:
https://docs.ansible.com/ansible/2.7/modules/na_ontap_svm_module.html#na-ontap-svm-module
https://netapp.io/2018/10/08/getting-started-with-netapp-and-ansible-install-ansible/

清除舊的硬碟上的 foreign aggregate

如果將其它cluster用過的硬碟換到另一個cluster時,會發現上面還有之前的Aggr記錄,可用以下方式清除

1.進入diag mode

NA-Cluster::> set diag

2.用disk show 確認舊的硬碟ID以及Aggr名稱,可以看到1.1.1 disk上還有之前的Aggr資訊

NA-Cluster::*> disk show

                     Usable Disk   Container Container

Disk                   Size Shelf Bay Type Type        Name Owner

---------------- ---------- ----- --- ------- ----------- --------- --------

1.0.21               3.63TB 0 21 FSAS    spare Pool0 NA-Cluster-N1

1.0.22               3.63TB 0 22 FSAS    spare Pool0 NA-Cluster-N1

1.0.23               3.63TB 0 23 FSAS    spare Pool0 NA-Cluster-N1

1.1.0                 3.63TB 1 0 FSAS   spare Pool0 NA-Cluster-N1

1.1.1                 3.63TB 1 1 FSAS   aggregate node1_root(1)

                                                                                             NA-Cluster-N1                                                                           

3.用storage aggregate remove-stale-record 清除舊的Aggr資訊

NA-Cluster::*> storage aggregate remove-stale-record -aggregate node1_root(1)
-nodename NA-Cluster-N1

4.用 storage aggregate show-spare 確認1.1.1已變成spare disk

NA-Cluster::*> storage aggregate show-spare                                                 

Original Owner: NA-Cluster-N1

Pool0

Spare Pool

                                                           Usable Physical

Disk                        Type RPM Checksum Size Size Status

--------------------------- ----- ------ -------------- -------- -------- --------

1.0.21                      FSAS 7200 block 3.63TB   3.64TB zeroed

1.0.22                      FSAS 7200 block 3.63TB   3.64TB zeroed

1.0.23                      FSAS 7200 block 3.63TB   3.64TB zeroed

1.1.0                        FSAS 7200 block 3.63TB   3.64TB not zeroed

1.1.1                        FSAS 7200 block 3.63TB   3.64TB not zeroed

                                  

Reassign ADP Data Partition Disk

1.需先進入advanced mode

用storage aggregate show-spare-disks確認目前spare狀況

VMw-1.49~VMw1.60的Owner為Node:cluster1-01

VMw-1.61~VMw1.72的Owner為Node:cluster1-02

2.從System manager建立新的Aggregates時也可看到兩個Node下各有12個硬碟

3.將VMw-1.61的data partition從cluster1-02移到cluster1-01

4.下storage Aggregate show-spare-disks可看到VMw-1.61的Owner已經移到cluster1-01

5. 從System manager建立新的Aggregates時可看到cluster1-01下變成13個硬碟,cluster1-02下變成
11個硬碟

增加 NetApp volume inode數量

1.用df –i確認目前volume inode 使用百分比,可看到目前test_volume inode使用量已到97%


2.確認目前volume inode上限數量
volume show –vserver <vserver name> -volume <volume name> -fields files


3.將volume inode上限提高到200000
volume modify –vserver <vserver name> -volume <volume name> -files <inode count>


4.確認目前volume inode上限已提升到200000

volume show –vserver <vserver name> -volume <volume name> -fields files

.

5.df –i可看到目前test_volume的 inode使用量已降到48%

Clustered Data ONTAP 設定 syslog server

1.建立syslog server destination,名稱為syslogserver,IP:192.168.128.101

2.確認syslog server設定成功

3.將warning以上等級的event log送到syslog server (可依需求設定不同等級)

4.確認event log設定,可看到warning以上等級的event log會送到syslog server